A Laboratory Evaluation of Unidata's Starflow Doppler Flowmeter and
MGD Technologies Acoustic Doppler FlowMeter (ADFM)
By
Tracy B.Vermeyen, P.E.
Water Resources Research Laboratory
Bureau of Reclamation
Denver, Colorado
email: tvermeyen@do.usbr.gov
Acknowledgments
This evaluation was funded by Reclamation's Water Measurement Research Program (project WR99.24). The Starflow equipment was loaned to Reclamation by Mr. Geoff Carrigg from Unidata America. Mr. Carrigg was very helpful in assisting with this evaluation and his agreement to loan Reclamation a Starflow system was greatly appreciated.. This evaluation was conducted because messrs. Brian Sauer (SRA-6414) and Mark Niblack (YAO-6020) were interested in both acoustic flowmeters and wanted an independent test of the two systems. Mr. Cliff Pugh, Technical Specialist, peer reviewed this report.
Introduction
The Starflow ultrasonic Doppler instrument and the ADFM are unique devices which measure water velocity, depth, and temperature and they are integrated with data loggers. They are the first of a new generation of ultrasonic flow measurement systems.
Both systems use digital signal processing techniques, and they are able to perform in a wide range of environments. They can be used to record flows in pipes, channels and small streams and operates in a wide range of water qualities from fresh water to sewage channels. These instruments represent two distinct types of Doppler instruments available to measure water velocity, they are:
- Incoherent or continuous Dopplers, like the Starflow system, send out a continuous signal with one transmitter and measures signals returning from scatterers anywhere andeverywhere along the beam with a receiver, see figure 1 and figure 2. The measured velocities of the particles are resolved to a mean velocity that can be related to a channel velocity at suitable sites. The Starflow system costs about $1,700.
- Coherent or profiling Dopplers, like the ADFM, transmit encoded pulses with the carrier frequency along four beams and are able to target specific locations (depth cells), and only measure these reflected signals, see figures 2 and 3. This allows the velocity distribution in a water column to be profiled. These instruments are generally more complex and expensive when compared to incoherent Doppler systems. The ADFM system costs about $17,000.
Figure 1. Schematic of Starflow incoherent velocity
measurement technique.
Doppler-based Velocity Measurement Technique - During a measuring cycle, a ultrasonic pulse is transmitted at a fixed frequency, called the carrier frequency. A receiver listens for Doppler frequency shifts in reflected signals from any targets moving with the water. A measuring circuit detects the frequency changes. A processing system accumulates and analyses these frequency changes and calculates a representative Doppler shift from the acoustic reflections received. Each Doppler shift is directly related to the water velocity component along the beam. This is a physical relationship and if you know the speed of sound in water you can compute the velocity of the reflector (which is considered to be representative of the velocity of the surrounding water). In general, the Starflow and ADFM instruments do not need calibration for velocity measurement.
Discharge Computation Technique - Water depth (or stage) is measured and is used with a stage-area relationship to determine the cross sectional area of the flow measurement section. This stage-area relationship or cross section shape (e.g. circular or trapezoidal) are programmed into the flowmeter as part of the site information. Of course, the accuracy of this relationship is critical to the accuracy of the discharge computation. Each flowmeter system attempts to measure the average channel velocity using Doppler-based velocity measurement techniques. The cross sectional area and the average velocity measured by the instrument are multiplied to obtain a discharge.
LABORATORY EVALUATION
The Facilities - A side-by-side evaluation of the Starflow and ADFM was conducted in a flume in Reclamation's Water Resources Research Laboratory in Denver, Colorado. The rectangular flume dimensions are 8.5 ft wide and 4 ft deep and 60 ft long. The acoustic transducers were placed in the center of the channel about 20 ft downstream from the inlet transition. The pumped flow capacity to the flume is about 30 ft3/sec. The depth in the flume are controlled by tailboards used to change the open area of the channel at the end of the flume. The tailboards were located 40 ft downstream from the measurement section.
A second test was conducted in the WRRL's canal automation model. The rectangular canal cross section is 12-in wide and 18-in deep. The flow was set using a variable speed pump and a calibrated propeller flowmeter. The propeller flowmeter was calibrated using a strap-on acoustic flowmeter and has an uncertainty of ±2 percent of the true discharge.
Testing Procedure - The first series of four tests were run over a range of flows and depths in a 8.5 ft wide flume. During each test, a Sontek ADV (acoustic Doppler Velocimeter) was used to measure an independent vertical velocity profile at the same location as the Starflow and ADFM. This ADV-measured profile was collected to determine the average velocity along the flume centerline to compare with the average velocity measured by the two instruments. Likewise, a staff gage was used to determine an independent measure of the depth at the measurement section. The average channel velocity was determined for each test by dividing the discharge by the cross sectional area. The average channel velocity is what the two flow meters need to accurately measure in order to calculate an accurate uncalibrated discharge.
An independent measure of the flowrate was made using the laboratory venturi measurement system which is accurate to ± 0.5%. This model also had flow delivered using auxiliary pumps and piping for tests 2 through 4. The flow through the auxiliary system was measured using a strap-on acoustic flowmeter which has an uncertainty of about ± 1- 2%. The combined discharge uncertainty for tests 2 through 4 was estimated to be ± 1.4% (based on a 2% uncertainty for the strap-on acoustic flowmeter).
The second test was done in the WRRL's canal automation model with the discharge and depth held constant for 5 hours and 30 minutes. The Starflow was programmed collected data as fast a possible (5 second scan rate). This test was run to see how stable the Starflow's depth, velocity and discharge readings were with time.
Tests - Five tests were conducted in this evaluation. Both flowmeters were programmed to store data every 1 minute. The ADFM collected 160 profiles which were averaged prior to logging the data. The Starflow was set with a scan rate of 15 seconds, with about 500 velocity measurements per scan. The average depth and velocity for the four scans per minute were stored in the Starflow's data logger.
A fifth test was done to examine the long-term performance of the Starflow in the WRRL's canal automation model. The model is 12-in wide and 18-in deep. The flow was set at 1.0 ft3/sec and the water level was set at about 0.975 ft deep. Starflow data were logged at 5-second intervals and the scan rate was set to 5 seconds. This test represents the noisiest condition with respect to velocity measurement because it has the shortest sampling period allowable.
Test Results - Table 1 and figure 4 contain a summary of the data collected during the first four tests including percent error in the Starflow and ADFM measurements relative to the laboratory measured values. Table 2 and figure 5 contain a summary of the data collected during the fifth test including percent error in the Starflow measurements relative to the laboratory measured values.
Test Duration - For the first series of tests the duration of data collection varied. This is an important factor because acoustic velocity measurements depend on many individual samples to compute an accurate average velocity. In other words, a single Doppler velocity measurement is a degree of uncertainty, but the average of hundreds or thousands of measurements results in an accurate measure of the average velocity for the measurement period. Consequently, the results for these tests depend on the duration of the test and this showed up in the results. The duration of data collection for tests 1 through 4 were 60, 39, 20, and 50 minutes, respectively. This sampling length explains the difference in the ADFM's discharge uncertainty as presented in table 1. The ADFM's discharge uncertainty was within ±3 percent of the laboratory discharge for the long duration tests and significantly greater than ±8 percent for shorter duration tests, namely tests 2 and 3. The average ADFM uncertainty over all four tests was +1.33 percent.
The discharge uncertainty for the Starflow meter appears to be inversely proportional to the test duration, which is a very peculiar characteristic. The Starflow's discharge uncertainty was greater than +18 percent for all four tests. The average uncertainty in Starflow discharges over all four tests was +26.2 percent.
The Starflow instrument systematically over predicted the depth and velocity for all tests. The ADFM systematically under predicted the depth, but the error in average velocity varied widely from test to test. Figure 6 shows a bar graph of the comparison of discharge, depth and velocity for the four flume tests.
In the fifth test, the Starflow instrument systematically over predicted the velocity. However, the stability of the measurements were very good over the 5 ½ hour test. The data collected during this test are shown in figure 5. A summary of the averaged data and their percent error are shown in table 2.
Depth Measurements - For the first four tests, both transducer assemblies were mounted to the marine-grade plywood floor which is level is both directions. The stage measurements were collected to the nearest 0.005 ft at the wall of the flume at the measurement cross section. The reported resolution of the Starflow's pressure sensor, which is used to measure depth, is 0.003 ft (1mm) with an uncertainty of ±0.25%, up to a depth of 3.3 ft or 0.008 ft. The Starflow depth measurements were on average 1.37% greater than the staff gage measurements. This discrepancy in depth measurement does not perform up to the specified depth measurement accuracy.
The ADFM's depth measurement operating range is 0.5 to 16.4 ft. The specified long-term uncertainty is 0.5% ±0.033 ft. While the ADFM's uncertainty is greater than the Starflow's, the two performed similarly. The ADFM depth measurements were on average 1.14% less than the staff gage measurements. The ADFM depth sensor performed within the specified depth measurement accuracy.
Depth measurement is important for computing the discharge and the Starflow and ADFM meters performed this task with an average uncertainty of +1.37 and -1.14 percent, respectively. Figure 6 shows how the depth measurements compared to the staff gage measurements for all four tests. Both of these errors were systematic in nature which allows them to be corrected for in post processing of the data, if the systematic error is quantified by field calibration.
For the fifth test, the long-term depth reading was very stable (see figure 5). The average depth was 0.975 ft with a standard deviation of ±0.003 ft. A staff gage reading was not taken for comparison during this test.
Depth measurements were made for calm (wave less) water surfaces. Waves will add a degree of uncertainty in the depth measurements for both flowmeters.
Velocity Measurements - Velocity measurements were collected in the center of the flume using the Starflow, ADFM, and the Sontek ADV. The ADFM and ADV collected velocity profiles while the Starflow computes an average velocity. The reported resolution of the Starflow's velocity measurement is 0.003 ft/sec (1 mm/sec) with an uncertainty of ±2% of the measured velocity. The range of accurate velocity measurement is reported to be from ±0.07 to ±14.8 ft/sec (bidirectional). Tow tank tests [in Australia (Chalk 1995) and by the USGS (Laenen 1997)] confirm the Starflow's accuracy claims, but they identified a problem measuring velocities less than 0.07 ft/sec.
It is important to understand that the velocity calibrations for the Starflow instrument were done in tow tanks. For this calibration, the transducer is moving at a constant velocity and the water and acoustic scatterers are stationary. As a result, the velocities measured are not part of a velocity profile, but are constant with respect to the towed transducer. This type of calibration is not representative of open channel or pipe flow where the velocity changes with distance from the boundary. As a result, to ensure accurate discharge measurements the user has to develop a calibration relationship between measured velocity and the average channel velocity. This means the average velocity has to be determined using another method, such as stream gaging. This point is made in the Starflow manual, but they also say "Starflow instruments do not need calibration for velocity measurement." Which in a strict sense is true for a tow tank test, but it is not true for purpose of measuring the average velocity in a channel or pipe.
The ADFM has a reported resolution of velocity measurement is 0.01 ft/sec with an uncertainty of ±1% ± 0.02 ft/sec of the measured velocity. The range of accurate velocity measurement is reported to be from ±16.4 ft/sec (bidirectional). The ADFM's velocity profiling range is 0.5 to 16.4 ft. MGD Technologies reported that calibration tests were conducted in a flume to determine the uncertainty in velocity measurements. The ADFM calibration test setup was similar to the WRRL facilities used for these tests.
The ADV has a reported resolution of velocity measurement is 0.0003 ft/sec with an uncertainty of ±0.5 of the measured velocity. The range of accurate velocity measurement is reported to be from ±8.2 ft/sec (bidirectional).
For the flume tests, the velocities measured by the Starflow and ADFM and the percent error in the measurement with respect to the ADV velocity are shown in table 1 and figure 4. Average velocity measurements are used to compute the discharge and the Starflow and ADFM meters performed this task with an average error of +28.7 and +5.8 percent, respectively when compared to the ADV measured value. A portion of this error is attributed to measuring the velocity in the center of the cross section where velocities are typically higher than the average.
When Starflow and ADFM velocities were compared to the average channel velocity computed using the known discharge and the cross sectional area, the average errors were 24.22 and 2.49 percent, respectively. These results are the most practical way to evaluate the two instruments, because this method is normally used to determine the average channel velocity.
For the fifth test the Starflow velocity was compared to the average velocity computed using the known discharge and the channel's cross sectional area as computed by the Starflow. The average error in the average velocity measurement was 24.7 percent, as shown in table 2. This result agrees closely with the error in the flume tests.
In my opinion, I thought the Starflow would perform better in the canal model because the acoustic beam covers a large area of the cross section. As a result, the instrument should measure a more accurate average velocity, but the velocity error was slightly worse (0.6 percent) than for the flume tests. However, because the error percentage for the five tests are very similar this indicates that the Starflow can calibrated. This error percentage will likely change from site to site, because of changes in water quality and flow conditions. Note: The Starflow's error percentage determined in this evaluation should not be used instead of a field calibration.
Discharge Measurements - As previously mentioned, discharge is computed using the velocity measurements and a depth-to-area relationship specified by the user. All of these tests were done in rectangular channels. As a result, the area calculation is straight forward and subject only to uncertainty in the depth and width measurements. The flume and canal model had measured widths of 8.5 ft and 1 ft, respectively. The width measurements were accurate to the nearest 1/16 of an inch. This represents an uncertainty in width of the flume and canal models is ± 0.06 and ±0.5 percent, respectively. This error is quite small compared to the errors in the depth measurements, as described earlier.
For both the flume and canal model tests, the Starflow consistently computed a discharge that was 24 percent larger than the known discharge. For the flume tests, the ADFM's average discharge error was within ±2 percent of the laboratory discharge.
Limitations - Limitations in this comparison which may impact the results of the side-by-side evaluation tests is that the two transducers acoustic signals and subsequent reflected signals might interfere with each other. The Starflow and ADFM transducers transmit acoustic pulses at 1.56 and 1.23 MHZ, respectively. However, a comparison of side-by-side tests and stand-alone tests in the canal automation model showed that there was a small difference between the two sets of velocity data measured by the Starflow. For example, for a flow of 1 ft3/sec in the canal model, the errors for the stand- alone and side-by-side velocity measurements were 24.8 and 22.3 percent, respectively. I did not do a similar analysis for the ADFM because it's velocity measurements were close to the computed average channel velocity.
The width to depth aspect ratio for the flume tests varied from 3.3 to 5.7, which is typical for small to medium sized canals, but the aspect ratio for large canals can be much greater than what was tested in the laboratory. MGD technologies claims that the ADFM accurately measures discharge for aspect ratios up to 10:1. For aspect ratios greater than 10:1, calibration tests are recommended to confirm ADFM's performance.
Water quality in the laboratory setting is much different than a field application. In our laboratory, the particle are primarily small air bubbles and miscellaneous debris. In the field, the particles will likely be sediment and aquatic debris which will likely have an impact on the performance of both the Starflow and ADFM systems. Acoustic flowmeter applications must take into consideration the water quality at the site for all seasons. For example, during spring runoff the sediment load may be substantially higher than later in the year. Sediment may bury the transducer during this time period. In the late summer, algae growing on the transducer or on the channel bottom may interrupt the acoustic signal. In both cases, maintenance will be required to keep the system operating properly. A system to place the transducer back on the bottom of the channel in the proper position is needed to make regular maintenance practical.
Temperature Measurement - Both transducer assemblies are equipped with temperature sensors to measure water temperature. Water temperature is a necessary measurement in order to compute the speed of sound in water. Neither manufacturer specified an uncertainty in their temperature sensors, but both sensors have a resolution of 0.1 deg. C. The operating range for the Starflow and ADFM temperature sensors are -17 to 60 deg. C and -5 to 35 deg. C, respectively.
For the first set of tests, both temperature sensors followed the changing temperature closely, but the ADFM's temperature sensor consistently measured a temperature 2 deg. C lower than the Starflow's temperature sensor. The average temperatures for the Starflow and ADFM sensors were 21.2 and 19.1 deg. C, respectively. I did not collect an independent temperature for this evaluation.
The difference between the two temperature measurements results in a difference in 6 m/s in the speed of sound in water. The speed of sound in fresh water at 20 deg. C is 1482 m/s. Consequently, a 6 m/s difference in the speed of sound represents a potential error of 0.4 percent in the velocity calculations.
For the second test, the long-term temperature reading was very stable. The average temperature was 21.47 deg. C with a standard deviation of ±0.03 deg. C.
Conclusions
The Starflow consistently computed discharges that were 24 percent greater than the known discharge for tests conducted in 1-ft-wide and 8.5-ft-wide rectangular flumes. The Starflow's velocity measurements were consistently 24 percent greater than the average channel velocity. While this over prediction in discharge is undesirable, the consistency in the percent error suggests that the Starflow should have a stable calibration over a range of flows and depths.
- The ADFM computed discharges that were on average within ±1.3 percent of the known discharge for a range of flows and depths in an 8.5-ft-wide flume. The ADFM's velocity measurements were, on average, within ±2.5 of the average channel velocity.
- This evaluation demonstrated that a Starflow flowmeter will probably require a site-specific calibration in order to verify the computed discharge in an open channel application. However, the ADFM performed well enough to be installed without an in situ calibration, if the accuracy requirements are within ±2 to 3 percent of the known discharge, if the width to depth ratio is less than 10:1. Accurate ADFM discharge measurements were made without any special consideration to the installation, aside from placing the transducer in the middle of the channel and aligning it with the direction of flow.
- Both instruments require their transducers be installed on the bottom of the open channel and maintenance to keep algae or other debris from collecting on the transducer. This maintenance will require a method to reinstall the transducer in the same position and alignment. This is especially important for a Starflow installation that has been calibrated for a specific location and transducer orientation.
- Variable water quality, sediment transport, and hydraulic conditions at the acoustic flowmeter site can impact the performance of both the Starflow and ADFM. However, the ADFM appears to be more robust in its ability to handle changes in water quality, sediment transport, and hydraulics.
- Depth and temperature sensors in both instruments performed as expected. The ADFM's acoustic depth sensor requires no maintenance; while the Starflow's pressure sensor uses a vent tube opened to the atmosphere. Maintenance of a desiccant-filled drying cannister is required to keep the vent tube dry and free from condensation. Depth measurements were made for wave less water surfaces. Waves will add a degree of uncertainty in the depth measurements for both flowmeters.
- Both systems require a communications cable between the electronics and the transducer. The cable will likely collect debris and will have to be cleaned. Debris build-up may cause the transducer to move. A solid anchorage for both systems is required to prevent movement. Likewise, vandalism may be another source of transducer movement.
- Both manufacturers claim that their acoustic transducers will not require calibration for "long-periods." However, this claim is difficult to substantiate without a costly calibration. Consequently, comparisons with another discharge measurement should be done regularly to verify flowmeter accuracy. As long as the transducer faces are clean and not damaged or worn they should work properly. The life of the waterproof cables are another issue which should be considered.
- Both systems are equipped with data loggers that were easy to program and download data. Both systems can be set up to work with SCADA systems, but this function was not evaluated.
Recommendation
Acoustic flowmeters are a new technology which are well suited for difficult flowmetering sites where traditional discharge measurement structures (weirs and flumes) are not practical. For example, sites with backwater problems caused by downstream gates and tides. These instruments combine, in a small package, the capability to measure depth, velocity, and temperature, and using this information calculate and log a discharge. Like all electronic systems, acoustic flowmeters require periodic maintenance which will vary from site to site.
I think the Starflow system has a niche in the discharge measurement market. It is capable of logging a continuous record of depths and velocities at a very reasonable price. The hidden cost of the Starflow system is the calibration cost. Depending on the accuracy required, the user should check the Starflow's discharge computation with an independent measurement as frequently as the user would normally stream gage the site until they are comfortable with flowmeter's accuracy and stability. After an acceptable calibration is established, stream gaging should be done monthly, or as frequently as needed, in order to have a record for quality assurance and quality control (QA/QC) purposes. Used in this way, the Starflow logs a continuous discharge record and eventually a reduction in the number of manual discharge measurements.
As previously mentioned, there are many factors which can affect the performance of the Starflow's velocity measurement and depth measurement. Consequently, each installation will have unique performance characteristics that may require more or less attention to address.
At sites which may require unacceptable levels of calibration, I would recommend spending the extra money for the ADFM system. The ADFM is more robust in its ability to accurately measure velocities in variable water quality and hydraulic conditions. For the same site, this system will usually require fewer calibration checks than the Starflow system.
Because this technology is new, I do not know how durable these systems are in field applications. Consequently, I would not suggest that a new user purchase several of these systems until the reliability and accuracy are established for a season in order to evaluate the system's ability to meet their specific water measurement needs.
Personal Communication with Starflow's Manufacturer
These tests generated many questions concerning the performance of the Starflow flowmeter. As a result, I sent several questions to Mr. Geoff Carrigg at Unidata America. The responses from Unidata do not affect the conclusions or recommendations in this report. The questions and Unidata's responses are list below:
- Question 1. Is an overprediction +24% of velocity typical for the Starflow in turbulent open channel flow?
Unidata's response, Yes, although a more typical channel calibration range is 10%.
- Question 2. How close to the bottom can the Starflow pick up Doppler shifts? In other words, what is the dead zone from the bed to the depth of the first processed scattered?
Unidata's response, Approximately 50mm.
- Question 3. Is a calibration always necessary for Starflow to accurately measure average channel velocities in an open channel?
Unidata's response, Uncalibrated, typical errors are up to 10% and depending upon the
application, can be significantly greater. Calibration is required if higher accuracy is desired.
- Question 4. Do you have any similar flume calibration data which contradict my findings?
Unidata's response, We do not have any test data of this type available.
- Question 5. Do you have any suggestions on how to mount your transducer so that it can easily retrieved and deployed to the same location, during maintenance? Many of our canals are rarely drained.
Unidata's response, We do not offer any mountings of this type. We are aware that customers have fabricated various mounting systems that allow Starflow removal for "full channel" maintenance but we do not have any details.
- Question 6. I am aware of Australian tow-tank and USGS tow-tank tests which confirm your velocity accuracy specifications. But I think tow-tank tests do not represent velocity profiles which occur in open channel and pipes. Do you have any comments on the difference between the Starflows performance in tow tank tests and flume tests?
Unidata's response, A number of significant differences exist in the conditions in a tow tank
versus a flume that will affect the velocity that the Starflow measures, particularly in regard to turbulence and velocity gradients.
- Question 7. Do you have a client list who are using Starflow systems in irrigation canals?
Unidata's response, Unfortunately we do not have a list.
References
Chalk, Anastasia, 1995, A review of the theoretical and practical calibration of the Unidata Starflow ultrasonic Doppler instrument, Water Resources Directorate Surface-water Branch, Instrument Facility Report, Welshpool, Australia.
Unidata, 1998, Starflow Ultrasonic Doppler Instrument with MicroLogger, User's Manual; No. 6241, Version 3, Revision E, Software, Model 6526B, published by Lynn MacLaren Publishing, Australia.
Laenen, Antonius, Information on Low-Cost Doppler for In Situ Measurement, US Geological Survey, WRD Instrument News, Issue No. 79, December 1997.
Carrigg, Geoff, Unidata America, Personal Communication (e-mail), May, 1999.
Figures
Last reviewed: 11/18/04