Fall 96 - Considerations for Data Quality and Uncertainty in Testing of Bicycle Aerodynamics
By: Michael J. Flanagan, Ph.D.
Testing of bicycle aerodynamics in wind tunnel facilities presents a unique
set of challenges generally not found in traditional aerodynamic testing.
For a typical aircraft test, a scale-model would normally be used and the
test results would be scaled and manipulated based upon a nondimensional
flowfield parameter such as Reynolds number - a term that relates velocity,
model scale, and the viscosity of air. Drag, the result of the
contributions of many elements such as shape factors, skin friction,
boundary-layer growth and transition, and shock waves, is highly dependent
upon Reynolds number. It is nearly impossible to simulate the full-scale
environment for typical aerodynamic testing (it would cost too much money
and require a power grid larger than the TVA); hence, some "skilled"
manipulation of the data is required - an effort that introduces
significant uncertainty in the accuracy of the test results.
Figure 1
For this test effort, it was possible to simulate the full-scale
environment. As the test article was full-scale and the wind speed was
matched precisely, duplicating the real-life flowfield was simple. With no
post-test manipulation of the data necessary, it was believed that highly
accurate drag measurements would be possible. Little did we know that, in
our attempt to measure highly accurate drag levels associated with bicycle
aerodynamics, we would uncover a Pandora's box of data quality issues
related to the very nature of cycling dynamics. The nature of this testing
did bring to light a peculiar point of view in the age -old quest of
accuracy versus precision. Simply put, even though a test can be organized
and run utilizing state-of-the-art equipment giving precise answers to 0.05
lb, the flowfield can be so complex given such variables as wheel rotation,
pedaling speed (cadence), and rider posture that accurate measurements are
impossible. Confused? Don't be! This is the age-old question of accuracy
versus precision. The answer you get is very precise - unfortunately, it
is precisely the wrong answer!
Since determining highly accurate drag levels was the primary focus of this
test, it was necessary to clearly understand the level of measurement
uncertainty of the test data. The term measurement uncertainty relates to
how far the measured answer is from the "true answer." Simply put, we can
think of the measurement uncertainty as the sum of the precision and bias
errors. The "state-of-the-art" data acquisition system, model mount, and
"rolling-road" simulator enabled us to measure very precise answers; but
the basic answers were "biased" by such "errors" introduced by rider
position, wind-chill (January is a cold month to sit in a wind tunnel at 30
mph), pedal speed, and wheel rotation ... to name a few.
To date, a significant portion of the known experimental data comes from a
technique referred to as "coast-down testing."1 This method involves the
derivation of bicycle (and rider) drag from the velocity achieved at the
end of a ramp of known geometry. Typical drag levels at 30 mph are of the
order of 6.5 to 8.0 lb for a rider on a racing bicycle in a racing posture.
2 This particular speed, bicycle, and posture conditions are chosen to
provide a degree of consistency for presenting test results. A drag
reduction on the order of 10 percent would save an estimated 2 minutes
total time during a typical 60-minute race. As races are won by mere
seconds, serious cyclists would gain an advantage with drag reductions on
the order of 1 percent, or 0.08 lb. Proponents of the coast-down technique
criticize the lack of adequate simulation during static testing of bicycles
in wind tunnels. Coast-down testing inherently produces time-averaged
results of the bicycle trajectory (the same type of results that would
result from static testing within a wind tunnel). The final velocities
derived from coast-down runs are affected by rider position, pedaling
technique, and the mechanical condition of the bicycle, as well as
environmental factors. Despite these influences, the uncertainty of the
drag levels associated with these tests have been quoted to within
approximately 0.02 lb. 3
EXPERIMENTAL SET UP
Four rider postures were considered as the controlling variables for this
test effort. Two typical postures, seenleft, reflect a broad sampling of
bicycle riders and uses. These body postures range from those used by
recreational riders to those used by competitive racing cyclists. The four
postures are described as: (1) general touring pose,(not shown) (2)
drop-touring pose, (3) racing pose, and (4) long-distance racing pose (not
shown) (similar to those popular with current triathletes).
Posture 2
Posture 3
The balance assembly used to measure axial force and hence drag force is
mounted in the seat post directly beneath the rider. This was done not
only in an attempt to minimize the effect of pitching moment on the balance
but also to constrain the bicycle (and rider) in pitch (up and down) and
yaw (left and right). The bicycle and rider, suspended within the test
section, are integrated with the centerline sting-support in the wind
tunnel. The heart of this balance is a one-component load cell utilizing
linear bearings to transmit all axial loads to a compression load cell.
Figure 3
The roller system, submerged beneath a false floor, enables the rider to
power the bicycle wheels at a speed that matches the tunnel airspeed and
simulates true motion of the rider and bicycle environment. This assembly
consists of a single roller for the front wheel and a dual-roller assembly
for the rear wheel. The rear wheel of the bicycle transfers power to the
dual rollers. A belt drive connecting one rear roller to the front roller
enables the front roller to spin the front wheel at the same speed in order
to match wheel rotational velocities. A simple speedometer connected to
the front wheel hub enables the rider to match wheel rotational speed, and
there-fore bicycle velocities, to the tunnel velocity. The entire test
set-up is shown above in Figure 3.
Part of the process to minimize the uncertainty involves the proper
determination of interference effects, also called tares. These values
cannot be overlooked as they can cause the drag levels to vary by as much
as 100 percent! Consideration must be given to the effect of the floor
contact on the overall drag level. As the cycle wheels "bridge," or
interfere with, the drag balance at the three roller-contact points, the
interference effect must be measured and subtracted (or added) to the final
measurements. Essentially, every data point must be measured several times
with the wind off in order to determine these tare values. The wheels are
in contact with a nonmeasurement surface. The extent of that contact is
dependent upon a complicated relationship between tire deflection, sting
deflection due to model weight, and center-of-gravity position, as well as
the load-cell deflection. Not all of the individual tares act in the same
direction. Careful attention to sting deflection and tire deformation is
necessary to ensure that the tires contact the rollers, yet do not hinder
axial movement of the bicycle due to drag.
EXPERIMENTAL RESULTS
Test results give time-averaged drag levels on the order of 10.2 lb. for a
rider in racing posture 3, at 30 mph. This drag level increases to about
12.9 lb when the rider moves to a more vertical position, one that is
typical of a posture for a recreational cyclist. These levels are higher
than the drag levels quoted in all three references by approximately 20
percent. This variation is understandable considering the rider accounts
for almost 70 percent of the total drag and the rider for this study was of
muscular build - not typical of the average racing cyclist. While
steady-state test results are similar to those found in previous works, in
truth, the basic nature of this drag data is highly dynamic - a fact that
will introduce bias errors during time-averaging. These time-dependent
trends were not evident in the test results from the "coast-down"
technique.
Figure 4
The graph, Figure 5, presents a comparison between the drag levels while
the rider is pedaling at a cadence of 75 rpm versus not pedaling
(stationary). Notice the dynamic characteristics of the data. The mean
(average) value of the data at a cadence of 75 rpm is approximately 10.2 lb
with a corresponding root-mean-square (RMS) drag value, a bias error, of
approximately 0.64 lb. These data for posture 3, at a wind speed of 30
mph, graphically demonstrate the effect of pedal cadence. The difference
between the mean drag values of the two curves, pedaling versus stationary,
is approximately 4.0 lb. In addition, the results from coast-down testing
fall between these traces. The second plot, in Figure 6, presents a
comparison of drag levels for postures 1 and 3. Notice how the data are
quite similar in dynamic content. The difference in average value is due
to the rider position (exposed frontal area). It is particularly
significant to note that the difference between the mean drag values for a
rider pedaling and for the pedals stationary is independent of body
posture. This drag value difference seems to be a constant value of
approximately 4.0 lb.
Figure 5
Figure 6
MEASUREMENT UNCERTAINTY
For the -25 to +25 lb range load cell used in this study, the manufacturer
quoted accuracy is 0.2 percent full-scale, or about 0.05 lb. Precise
calibration within the range of 0 to 15 lb provides for an expected
precision of 0.03 lb. Careful attention to the calibration, the data
acquisition system, and electrical interference (noise) provides for an
overall precision error that is approximately 0.05 lb. These values can
generally be determined prior to the test. Determining the bias error
however, is not so simple and generally must be done after the test has
been completed.
Data repeatability is also a significant problem. Subtle variations in
body position while trying to maintain a specific posture led to variations
in the mean drag values by as much as 1.4 lb. Additionally, pedal cadence
had a strong influence on RMS drag value. This was seen when the author,
at a cadence of 55 rpm, managed to produce an RMS drag value of almost 1.5
lb (almost three times the value at a cadence of 75 rpm). Finally, the
windchill factor was not an insignificant problem.
Determining the bias error generally requires a detailed statistical
analysis of the test data. Allow me to digress momentarily to describe some
important basics of statistical analysis (understand, of course, this paper
was written by a rocket scientist). When I talk about statistical analysis
I refer to identifying trends buried within the data. These trends are
evident from four statistical terms named: mean (average) value; variance
(standard deviation or RMS), skewness (symmetry); and kurtosis (deviation
from a Gaussian "bell" shape). Drag levels are calculated using the first
two statistical moments, the mean and RMS values. These two terms provide
a good approximation of the average value and the relative variation within
the data - the lower the RMS, the lower the overall error. In order for
these values to have any physical significance, the mathematicians who
derived these relationships required that the flowfield would be ergodic.
Okay, so what does it mean when a flowfield is ergodic? Well, ergodicity
is a property whereby, for all probability, all data statistics can be
calculated from a single data-trace by time-averaging. More simply put, the
average of 1000 data points taken during one run would be the same as the
average of 10 data points from each of 100 runs. This concept of
ergodicity is crucial to wind tunnel testing as it allows scientists to
accurately draw conclusions from a small set of data. If each data point
needed to be repeated, say 10 times, the cost of testing, would be
prohibitive.
Figure 7
This concept of ergodicity requires that the data be stationary,
specifically, that the data exhibit no time-dependence in any of the
statistical moments. Figure 7 presents a plot that is representative of
the RMS drag value (second statistical moment) versus time. For this
particular run, 20 seconds of data were taken corresponding to
approximately 10,000 data points. The result of this run is one "test
point" - the average of all 10,000 data points for this run. The data
exhibit a trend to converge to approximately 0.8 lb - the value is actually
meaningless. The very fact that this plot shows ANY time dependence, as
opposed to being a straight line, is VERY significant. This plot shows
that the flowfield is NOT ergodic. Consequently, the entire data set is
suspect from an accuracy standpoint. The concept of ergodicity requires,
regardless of the particular ensemble (data set) analyzed, that there will
be no time-dependence in the ensemble - that all test points will yield
identical results. In order to achieve an accurate result, repeat runs
would be necessary to achieve some type of ensemble average. This ensemble
average would provide a more accurate estimation of the "true" drag level.
How many repeat runs are necessary for this one test point? The only way
to know for sure would be to make perhaps a dozen or more and plot the
average value from those runs.
One can now understand the problem encountered. If our original test
matrix involved several model configurations at several velocities, 100
test points could be involved - say five days of testing at $4000/day.
Now, if we are required to repeat each test point 10-15 times to achieve
our desired accuracy goals, we now need 50-75 days and $300,000.
CONCLUSIONS
The most significant trends apparent within the test data are the dynamic
influence of wheel rotation and the rider pedaling. Additionally, the
effect of subtle changes in body position during the run is significant.
While the dynamic trends are of interest, the most useful information to
the rider is some type of accurate, time-averaged drag value. Accurate
time-averaging, however, does require the assumption of flowfield
ergodicity. As the flowfield surrounding the cycle and rider within the
wind tunnel is clearly NOT ergodic, "real" drag values are meaningless for
short-duration testing. In order to achieve meaningful drag levels, the
test plan must allow for sufficient repeat runs for each test point.
What are the reasonable limits to data quality? For this test, the
customer wished to know drag levels to 0.06 lb or approximately the
predicted measurement uncertainty level. This was not possible after the
fact, as the flowfield was clearly not ergodic and the test plan and
funding did not allow for any additional testing. Some of the information
obtained was useful. In particular, specific effects during a given run,
such as the effect of head-up versus head-down, provided a repeatable
increment from run to run. So, even though the absolute level of the drag
was never determined, a highly accurate measurement could be made of the
effect of handlebar droop, head-up versus head-down, and pedaling versus
stationary. In addition, as the effect of pedaling was the primary reason
the flowfield was not ergodic, all data taken while the rider was
stationary data were valid.
For this particular technique, the sources for uncertainty are neither the
instrumentation nor the data acquisition system. Rather, the close-coupled
nature of the balance and test vehicle system creates a flowfield
environment that is not ergodic. The presence of the rider and the
dynamics due to the pedaling required to achieve proper flowfield
simulation force the assumption of ergodicity to be invalid. This raises
serious doubt about the certainty of the measured drag levels. Only through
repetition of one particular test condition will sufficient data be
available for an ensemble average. This will provide a more accurate
estimation of the "true" drag level.
References
1. Schondorf, P. (1982) Design Examples - Coast Down Tests. In Proceedings from the First International Human-Powered Vehicle Scientific Symposium.
2. Phone conversation with John Cobb of Bicycle Sports; Shreveport, LA.
3. Kyle, C.R. (1982) Improving the Bicycle Racing System. In Proceedings from the Second International Human -Powered Vehicle Scientific Symposium.
Michael J. Flanagan, Ph.D.
University of Dayton Research Institute
Dayton, OH 45469-0117
email: flanaginj@elwood.udayton.edu