Search by Catalog Number
4538 Westinghouse Street
Ventura, CA 93003
Deception of the Douglas Bag Validation Method
What is the Gold Standard of Incompetence?
By Andrew Huszczuk Ph.D. Edited by John Hoppe
The reason for this paper is misinformation being fed to unsuspecting prospective buyers of metabolic measurement system. These systems are known generically as Metabolic Carts (MC), CPX systems, Ergospirometry systems in Europe and also as calorimetry measurement systems; for simplicity, we will hereafter refer to them as MC's.
It appears that some manufactures of MC's base their claim of superiority on a statement that their instruments are either calibrated or validated using the Douglas bag method.
We challenge the validity of such claims.
Would you take a medication knowing that a pharmacy used an uncalibrated scale to weigh its ingredients? Would you board a plane knowing that the fuel or altitude gauges are not calibrated at frequent intervals?
In these and thousands of other applications scientific bases and rules of metrology must be obeyed to assure chaos-free operation of modern societies. To scrutinize performance of measuring devices a process of calibration must be carried out by means of applying a known standard and getting back a correct reading.
Whereas a single physical entity (e.g. mass, temperature, pressure etc.) measurement involves a relatively simple calibration procedure, more complex systems utilizing multifactorial sensing require a two-stage calibration. First, all sensing components have to be calibrated separately by applying sensor-specific standards (e.g. certified gas mixtures of a known composition) and performing fine-tuning to obtain corresponding readings. These readings, in turn, are used to "scale" the calibration function of software.
The second stage of calibration has to verify the final performance of the measurement system as a whole. This necessitates generation of known quantities and qualities of the measurement-specific variables (i.e. the simulation of exhaled gas mixtures), which, to become standards, have to be prepared with sufficient accuracy and delivered to the system under calibration in a typical dynamic pattern simulating the "real life" situation.
Most of such complex measurement systems can be classified as mass flowmeters.
In medical applications mass flowmeters are used to measure the level of metabolism at rest and during incrementing intensity exercise. They are called metabolic carts (MC's), and essentially measure the oxygen consumption (VO2) and the carbon dioxide output (VCO2) of the body, usually expressed in liters per minute STPD. These two variables are usually complemented by minute ventilation of the lungs (VE) and heart rate (HR) expressed in liters per minute and beats per minute respectively. From the above four variables a score of additional ones can be derived to enhance the process of diagnosis in fields such as cardiopulmonary medicine, sports medicine and physiology, metabolic and nutritional disorders, geriatrics, rehabilitation, occupational medicine, assessment of fitness etc.
Current State of Technology
All commercially offered metabolic measurement systems require initial calibration of respiratory (usually expiratory) flowmeters and a two-point calibration of the O2 and CO2 analyzers. Respiratory flowmeters are most commonly calibrated with a large (usually 3-liter) syringe manually powered to displace known volume of air through them using slow, medium and fast strokes. Since, in metabolic measurement applications, the minimal dynamic range of flows expected to be sensed by flowmeters is 0.25 to 3 L/sec (a twelve-fold span) and in athletic applications the upper flow range should be at least 5 L/sec (a twenty-fold span), the manually induced puffs or gusts of air are inadequate to assure better than Â± 5% accuracy across a desired range due to the following reasons:
Similar issues of non-linearity affect performance of gas analyzers. Fluctuations of sampling flows, barometric pressure, ambient temperature and humidity will induce drifts which, when unchecked, may exceed the capability of the "on-board" (i.e. being an integral part of analyzers) linearizing devices. Therefore, the two-point, usually air for low CO2 and high O2 and certified standard gas mixture for low O2 and high CO2, calibration of gas analyzers has limited validity as it determines only an average slope of the output signals versus gas concentrations relationship. The actual point-by-point relationship does not exactly follow the straight lines joining two respective points that span the whole measurement range for each gas.
Finally, sloppy or incorrect handling of parameters* greatly, and often overwhelmingly, contribute to the overall performance accuracy because they affect correction factors between physical conditions (ATPS, BTPS, and STPD) or the manner of mass flow assessment computation:
Incredibly, but the foregoing long list of potential and usually real sources of errors is commonly ignored as very few metabolic equipment manufacturers bring up these issues and relevant scientific organizations do not offer recommendations to standardize the process of calibration. Consequently, currently used metabolic measurement equipment rarely performs within boundaries of decency i.e. Â± 5% of accuracy and often approaches or exceeds Â± 10% without notice or complaints.
* see footnote on the last page
When sales representatives of some manufacturers are confronted with issues of accuracy they may claim that their systems are validated with the Douglas bag method, which they dare to call a "gold standard" of calibration.
A beginning of 20th Century method - the gold standard?
Moreover, some manufactures actually use this claims in advertising. One may not expect much competence from sales representatives, but advertising represents official claim of a manufacturer and, as such, is alarming. Perhaps the use of the Douglas bag method for validating metabolic measurement systems should more appropriately be called "The Gold Standard of Incompetence".
Validation used as euphemism for calibration using bag collection of exhaled gases is not and can not epitomize the very essence of calibration for the following reasons:
In practice, even when perfectly executed the Douglas bag method yields results within a range of Â± 5% accuracy, which can be assessed only when the volumes of CO2 and O2 (deficit) are known prior to commencement of bag collection. Here is how one can perform such an experiment:
Hopefully it wasnâ€™t too discouraging, but remember, you did not have to deal with warm fully saturated gases of a real test situation. This simple test gave an idea for an ultimate calibration method consisting in combining strictly predetermined mass flow of CO2 and O2 (negative, to simulate consumption) delivered to the system under calibration by means of an intermittent breathing-like pattern of adjustable minute ventilatory flows (VE). By keeping metabolic flow constant, while adjusting VE, a practical range of flow and gas concentration spans can be scanned to detect ranges of non-linearity. This method is described in more detail on this web site in "Dr Andrew's Corner".
Metabolic measurement systems, as mass flowmeters, can be calibrated only by means of generating standard mass flows that can be set at will and delivered in a form of respiratory patterns. Existing software corrections (and in some cases "fudge-factors") are based on one-time analysis of any given transfer function and are not capable of effective dealing with unavoidable analog drifts.
Ultimately, we expect that the use of manually operated syringes and calibration gas mixtures will become obsolete to be replaced with advanced software that compensates all existing non-linearities by analysis of responses to forced mass flows and executes appropriate corrections across all operational spans within 3-5 minutes preceding every test.
* Incidentally, most of User Manuals and even many research publications negligently confuse term variable with term parameter. To reiterate: a variable is a dynamic quantity that has to be measured as it evolves (e.g.: expiratory flow, tidal concentration of O2 and CO2, heart rate, etc), whereas a parameter is a quantity that stays constant in the case considered, such as the current test, but affects computational outcome and may change for any other test conditions (PB, T, RH)