Start Submission

Reading: Possible malfunction in widely used methane sampler deserves attention but poses limited imp...


A- A+
Alt. Display

Policy Bridge

Possible malfunction in widely used methane sampler deserves attention but poses limited implications for supply chain emission estimates


Ramón A. Alvarez ,

Environmental Defense Fund, Austin, Texas, United States, US
X close

David R. Lyon,

Environmental Defense Fund, Austin, Texas, United States, US
X close

Anthony J. Marchese,

Department of Mechanical Engineering, Colorado State University, Fort Collins, Colorado, United States, US
X close

Allen L. Robinson,

Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States, US
X close

Steven P. Hamburg

Environmental Defense Fund, Austin, Texas, United States, US
X close


Estimates of methane emissions from natural gas production sites in the United States based on recent studies have been questioned due to possible malfunction of the Bacharach Hi Flow® Sampler (BHFS), the primary measurement instrument used for two out of five source types examined in those studies (equipment leaks and chemical injection pumps). Without assessing whether the BHFS malfunction occurred in those studies, we constrain the possible underestimation of emissions associated with the BHFS-based results by excluding potentially affected measurements. Assuming leak emission rates are similar for sites with low and high methane content, U.S. methane emissions from equipment leaks and chemical injection pumps in recent studies could be underestimated by up to 40–80% due to a malfunctioning BHFS. We discuss uncertainties associated with this estimate. While a 40–80% underestimation is important when characterizing individual source categories, the potential implications are attenuated when aggregating emissions across the five sources examined in the recent studies (<12–24%), across all sources in the natural gas production segment (<7–14%), or across the entire supply chain (<2–5%). Therefore, potential errors caused by BHFS malfunction in recent studies would not substantially alter estimates of methane emissions from the U.S. natural gas supply chain. The possible malfunction of such a widely-used instrument deserves further attention to ensure that its use in quantifying emissions from individual supply chain components is not compromised.
Knowledge Domain: Atmospheric Science
How to Cite: Alvarez, R.A., Lyon, D.R., Marchese, A.J., Robinson, A.L. and Hamburg, S.P., 2016. Possible malfunction in widely used methane sampler deserves attention but poses limited implications for supply chain emission estimates. Elem Sci Anth, 4, p.000137. DOI:
 Published on 09 Dec 2016
 Accepted on 25 Oct 2016            Submitted on 27 May 2016
Domain Editor-in-Chief: Detlev Helmig; Institute of Alpine and Arctic Research, University of Colorado Boulder, Boulder, Colorado, United States


The recent surge in U.S. natural gas and oil production, made possible by the use of horizontal drilling and hydraulic fracturing to tap hydrocarbons in unconventional geological formations like shale or tight sands, has been accompanied by concerns over potential climate, environmental, and health implications (Moore et al., 2014). These concerns were reinforced by reports (Brandt et al., 2014) that available inventories underestimated methane (CH4) emissions from the natural gas supply chain, which in turn stimulated an extensive body of new research to improve emission estimates.

A research team led by the University of Texas at Austin published three papers reporting measurements of CH4 emission sources at natural gas production sites, hereafter referred to as the UT Production Studies (Allen et al., 2013a, 2015a, 2015b). The UT Production Studies quantified emissions using a diversity of methods and instrumentation, among which was the Bacharach Hi Flow® Sampler (BHFS) (Bacharach Inc., 2015). Similar instruments have been used to measure CH4 emissions in the natural gas industry since the 1990s (Kirchgessner et al., 1997); the BHFS has been commercially available since 2001 (Howard et al., 2015). The performance of the BHFS under certain conditions has been questioned (Howard et al., 2015; Modrak et al., 2012), leading to concerns that the UT Production Studies may have substantially underestimated CH4 emissions (Howard, 2015a, 2015b).

The BHFS is a portable, battery-powered instrument packaged inside of a backpack; it was designed to quantify CH4 emission rates from leaking components common to natural gas operations, such as pipe fittings, valve packings and compressor seals and has a measurement range of 0.05–8 standard cubic feet per minute (scfm, equivalent to 1.4–230 L min-1) (Bacharach, Inc., 2015). The operating principles of the BHFS are described elsewhere (Bacharach, Inc, 2015; Howard et al., 2015; Modrak et al., 2012; Howard, 2015b; Brantley et al., 2015); here we provide a brief summary. The BHFS generates suction flow rates of 8–10 scfm to fully capture an individual component’s emissions along with surrounding air. Emissions are captured by placing a sampling hose, connected to one of a variety of sampling tools, immediately next to or around a leaking component. The instrument performs measurements at two constant flow rates differing by 20–30% to test whether the emissions from a source are being fully captured (complete capture can be assumed if emissions determined at both flow rates agree within ∼10%) (Bacharach, Inc., 2015). The BHFS determines a source’s emission rate by multiplying the CH4 concentration of the resultant gas stream passing through the instrument’s detector by the known flow rate through the sampler (the instrument applies a background correction for CH4 in ambient air using a parallel sampling and detector train) (Bacharach, Inc., 2015). It should be noted that the measured CH4 concentration inside the instrument is not the CH4 concentration of the leak source (which typically ranges from 70–98 mole% for natural gas sources); the resulting concentration is lower due to dilution with suctioned air. The BHFS quantifies CH4 concentrations with a dual-mode sensor designed to transition between a catalytic oxidation mode and a thermal conductivity mode depending on the CH4 concentration of sample/air mixtures within the instrument (0–5% CH4 and 5–100% CH4, respectively) (Bacharach, Inc., 2015; Modrak et al., 2012). The sensor is calibrated using CH4; in cases where the leaking natural gas source contains elevated amounts of non-CH4 species, calibrations using actual gas compositions or response correction factors are recommended (Bacharach, Inc., 2015). The non-CH4 fraction of natural gas streams exhibits significant spatial variability and includes other hydrocarbons like ethane, propane, butanes, and C5+ species, and may also include other species such as carbon dioxide, water, and hydrogen sulfide.

The BHFS has been reported to underestimate the actual CH4 concentration of sampled gas streams – and therefore CH4 emission rates of individual sources – because it can fail to transition from the catalytic oxidation mode to the thermal conductivity mode for reasons that have yet to be determined (Howard et al., 2015; Modrak et al., 2012; Howard, 2015b; Brantley et al, 2015). Howard et al. (2015) reported that the transition failure can occur when measuring natural gas emissions from sources with CH4 content less than ∼91% (mole percent). Howard (2015b) subsequently suggested the BHFS may underestimate emissions from high emitting sources when CH4 content is as high as 97%.

The possibility of BHFS sensor transition failure needs to be more closely examined. Until the underlying issues are better understood, users should proactively validate data collected with the instrument in order to reduce the risk of problems.

The objective of this paper is to constrain the magnitude of any possible underestimation of emissions resulting from the deployment of the BHFS in the UT Production Studies and discuss the potential impacts of such underestimation on the study’s primary conclusions and our understanding of the industry’s methane emissions more generally. We do not assess claims about the existence of sensor transition failure in the BHFS or whether it, in fact, occurred in the UT Production Studies. We note that primary co-authors of the UT Production Studies presented evidence (Allen et al., 2015c) leading them to conclude that sensor failure “did not significantly impact” measurements they made with the BHFS in both Allen et al. (2013a) and (2015a). Similarly, a recent study of natural gas compressor stations found no systematic bias between site-level emission estimates based on component-level measurements made predominantly with the BHFS and independent estimates based on a downwind tracer flux technique (Subramanian et al., 2015).


Potential malfunction of the BHFS would only affect a fraction of estimated emissions in the UT Production Studies. Specifically, measurements made with the BHFS were the primary basis for final emission estimates for only two of the five source categories examined: equipment leaks (“ELs”) and chemical injection pumps (“CIPs”) (Allen et al., 2015a, 2015b). Emissions from hydraulically fractured well completions and liquid unloadings were measured using other methods. While Allen et al. (2013a) reported emissions for pneumatic controllers (PCs) based on measurements taken with a BHFS, the UT Production Studies’ final estimate for PCs is based on the more systematic sample in Allen et al. (2015a) (600 Gg, Table 1); the latter relied primarily on a different measurement technique (only 12% of reported PC measurements in Allen et al. (2015a) were made with the BHFS). We do not assess the techniques used to quantify PC emissions in Allen et al. (2015a).


Table 1.

Comparison of methane (CH4) emission estimates from the U.S. natural gas supply chain: (i) as reported in the 2014 U.S. EPA Greenhouse Gas Inventory (GHGI) (U.S. Environmental Protection Agency, 2014); (ii) 2014 GHGI, adjusted to include recent source-specific measurements led by the University of Texas at Austin (UT Production Studies) (Allen et al., 2013a, 2015a, 2015b); and (iii) sensitivity of (ii) to potential emission increases of 40% and 80% to account for possible sensor transition failure in the Bacharach Hi Flow® Sampler (BHFS) if it had systematically affected all of the UT Production Studies’ measurements of ELs and CIPs below 91% and 97% CH4 content, respectively (adjusted emissions shown in italics)

Emission Sources Annual CH4 Emission Estimate (Gg)
2014 US EPA GHGI (Net 2012 Emissions) (U.S. Environmental Protection Agency, 2014) UT Production Studies, as Reported in Allen et al. (2015a)c Sensitivity test: 1.4x BHFS-based measurements (% increase over Allen et al. (2015a) Sensitivity test: 1.8x BHFS-based measurements (% increase over Allen et al. (2015a)
A. Equipment Leaks (“ELs”)a 189 307 430 553
B. Chemical Injection Pumps (“CIPs”) 65 73 102 131
C. Hydraulically Fractured Well Completions & Workovers 217 24 24 24
D. Pneumatic Controllers 334 600 600 600
E. Liquid Unloadings 274 270 270 270
F. Other Production Sources not Measured in UT Production Studiesb 913 911 911 911
Subtotal - Sources measured in Allen et al. (2013a) with BHFS (A–B) 254 380 532 (40%) 684 (80%)
Subtotal - All sources measured in UT Production Studies (A–E) (Allen et al., 2013a, 2015a, 2015b) 1,079 1,274 1,426 (12%) 1,578 (24%)
Subtotal - All Sources in GHGI Production Segment (A–F) 1,992 2,185 2,337 (7%) 2,489 (14%)
Subtotal - Other GHGI Segments (Transmission and Storage, Processing, and Local Distribution)b 4,194 4,194 4,194 4,194
Total - Natural Gas Supply Chain 6,186 6,379 6,531 (2%) 6,683 (5%)

To constrain the possible underestimation in national CH4 emissions from ELs and CIPs due to sensor transition failure in the BHFS, we analyzed Allen et al. (2013a) data from 278 individual measurements of ELs made with the BHFS (Figure 1) (Allen et al., 2013b). We simulated the potential underestimation by assuming that the BHFS systematically malfunctioned when measuring emissions from every EL and CIP with fractional CH4 content (%CH4) below fixed thresholds and then recalculated results after filtering out all potentially affected measurements. We considered two alternative thresholds of the %CH4 in natural gas below which sensor transition failure could occur. The lower threshold is based on the Howard et al. (2015) observation that sensor failure can occur if the %CH4 of the natural gas emission source is less than ∼91% (mol%). The upper threshold is based on additional analysis in Howard (2015b) that suggested the BHFS may underestimate high emitting sources when %CH4 is as high as 97%. Because BHFS failure has not been reported for sources with %CH4 >97%, we assume such measurements are unaffected.

doi: 10.12952/journal.elementa.000137.f001.
Figure 1.  

Measurements of CH4 emissions from equipment leaks at natural gas production sites as reported in Allen et al. (2013a, 2013b).

Dashed vertical lines at 91% and 97% CH4 content show thresholds associated with the reported, possible malfunction of the Bacharach Hi Flow® Sampler (BHFS) used to make these measurements (Howard et al., 2015; Howard, 2015b). Summarized above the figure are our calculations of the mean emission rate of the complete, unfiltered dataset (0.11 scfm) and of the subsets obtained by filtering out measurements below 91% and 97% CH4 content (0.15 and 0.19 scfm, respectively). Because BHFS malfunction has not been reported to occur for sources with CH4 content >97%, our analysis assumes such measurements are unaffected. We thus use the 80% difference in mean CH4 emissions between the full dataset and the filtered subset >97% CH4 content (i.e., 0.19 vs 0.11 scfm) to constrain the potential impact of BHFS malfunction on the results of BHFS-based measurements in Allen et al. (2013a). Symbols indicate the region where measurements were made: AP=Appalachian (circles); GC=Gulf Coast (squares); MC=Midcontinent (triangles); RM=Rocky Mountains (crosses). Vertical axis units are standard cubic feet per minute (scfm).

More specifically, we constrain the possible error introduced into the UT Production Studies due to BHFS sensor transition failure by comparing the mean emissions of all 278 ELs measured in Allen et al. (2013a) to the average of the subsets of ELs with %CH4 >91% and >97% (Figure 1). If sensor transition failure had systematically affected every measurement within the subset below a fixed %CH4 threshold, then the mean of the subset of measurements above the threshold could be considered to be characteristic of the full dataset (since those measurements would presumably have been unaffected by the sensor problem). We discuss uncertainties associated with this approach in the Discussion section.

Finally, we examined the sensitivity of national emission estimates to increases of 40% and 80% in emissions from ELs and CIPs (Table 1, rightmost columns). This sensitivity range is intended to constrain the potential error caused by systematic sensor transition failure in the BHFS in the UT Production Studies and is based on our analysis of the Allen et al. (2013a) EL measurements, described above.


The UT Production Studies quantified CH4 emissions from five sources or activities common to natural gas production sites (Table 1, Categories A-E). These five categories account for 50–60% of total estimated CH4 emissions in 2012 from the entire natural gas production segment (Allen et al., 2015a; U.S. Environmental Protection Agency, 2014). The final emission estimates for only two categories (ELs and CIPs) were exclusively determined from BHFS-based measurements. ELs and CIPs account for only 30% of the total reported emissions from the five sources assessed in the UT Production studies and 17% of total estimated emissions from the natural gas production segment (Table 1, column 3). Howard (2015b) reports higher values of 98% and 41%, respectively, but he did not account for updated estimates to Allen et al. (2013a) presented in Allen et al. (2015a, 2015b).

The UT Production Studies report lower CH4 emission rates for ELs having lower fractional CH4 composition (Figure 1). It has been hypothesized that these results are flawed because of a sensor malfunction in the BHFS that may occur for emission sources with fractional CH4 composition below 91–97% (Howard, 2015a, 2015b). To constrain the potential underestimation that such a malfunction could have caused in the UT Production Studies, we performed a sensitivity analysis by applying the mean emission rates of ELs with higher fractional CH4 composition (>91–97% CH4) to the entire EL dataset and determined the resulting difference in total CH4 emissions (Table 1, columns 4 and 5).

If BHFS sensor transition failure had occurred at every site with <91% CH4, then the mean of emissions from ELs may be up to ∼40% higher than the value reported in Allen et al. (2013a) (0.15 vs. 0.11 scfm CH4; see Figure 1). The corresponding value if BHFS failure occurred at all sites with <97% CH4 is ∼80% higher (0.19 vs. 0.11 scfm CH4). For comparison, Allen et al. (2013a) reported a relative uncertainty of ±36% (95% confidence level) for the mean emission rate of U.S. ELs.

Increasing Allen et al.’s (2015a) national estimates of CH4 emissions from ELs and CIPs by 40% and 80% to account for the possible underestimation due to BHFS sensor transition failure would increase the aggregate emissions from the five measured categories in the UT Production Studies by 12% and 24%, respectively (Table 1). After accounting for emissions from sources not measured in the UT Production Studies (using data from the EPA Greenhouse Gas Inventory) (U.S. Environmental Protection Agency, 2014), the potential increase in total emissions from the natural gas production sector is 7–14%. Moreover, the potential increase in natural gas supply chain emissions (production through local distribution) is 2–5%. We emphasize that these estimates are only intended to constrain the potential underestimation if BHFS malfunction had systematically affected every measurement of ELs and CIPs in Allen et al. (2013a) with %CH4 <91–97% – they should not be interpreted to represent estimates of actual error in previously published results. In sum, the analysis is intended to provide context for interpreting concerns about BHFS problems on the major conclusions from the UT Production Studies.


Plausibility of constraining assumptions

In this section, we discuss why our assumptions are plausible and lead to a reasonable constraint on the potential error caused by BHFS sensor transition failure in the UT Production Studies. Our constrained error estimate assumes: (i) BHFS sensor transition failure is the sole cause of the lower mean of ELs with <91–97% CH4 content (when compared to the mean of ELs >91–97% CH4); and (ii) the emission distribution of ELs >97% CH4 yields a reasonable upper bound for what the true mean of ELs would be in the absence of BHFS sensor transition failure. We are unable to assess these assumptions against empirical data because no comparable national datasets exist and almost all published EL emission rates are based on BHFS measurements; hence they may have been impacted by sensor failure. We acknowledge that these assumptions result in an indeterminate uncertainty and that further work is warranted to reduce it (for example, our assumptions could be tested with new datasets of EL measurements from sources with high and low %CH4 content across multiple regions). At a minimum, our analysis helps inform the issue.

The first bounding assumption is based on Howard’s (2015b) conclusion that the relative absence of measurements >0.4 scfm in the subset of EL measurements <97% CH4 (Figure 1) is caused by BHFS sensor transition failure. However, examination of the EL measurement data in Allen et al. (2013a) suggests that factors such as differing practices among natural gas producers (“producers”) may also play a role. Of the four regions surveyed for ELs in Allen et al. (2013a), the Appalachian region had the highest mean emission rate and the highest %CH4 (94 of 100 ELs had >97% CH4 and all were >95% CH4). In turn, the Appalachian accounted for 11 of the 15 ELs measured across all regions with raw BHFS readings >0.4 scfm. Remarkably, ten of the 11 ELs in the Appalachian region >0.4 scfm were at six sites belonging to a single producer (five producers provided 47 sites in the Appalachian). Excluding the ten values >0.4 scfm attributable to that single producer would reduce the mean of Appalachian ELs from 0.18 to 0.071 scfm CH4, a value intermediate to those observed in the regions with lower %CH4 (0.024, 0.082 and 0.093 scfm CH4 in the Rocky Mountain, Midcontinent, and Gulf Coast regions, respectively). Other factors like local regulations, equipment age, natural gas and oil production levels, and type of equipment could also lead to regional differences in emissions (Supplemental Material Text S1 contains analyses suggesting that lower absolute emission rates at sites with lower %CH4 may be partially attributable to lower production rates). Different producer practices and other factors that could contribute to the data patterns in Allen et al. (2013) deserve further scrutiny. The fact that other factors can potentially contribute to the observed differences in emissions reduces the likelihood that the maximum underestimation in the UT Production Studies due to BHFS failure alone is as large as our constrained estimate.

The second bounding assumption relies on the expected absence of BHFS malfunction for the subset of sources with >97% CH4. Specifically, we assume that the mean of emissions from the subset of EL measurements for sources >97% CH4 represents an upper bound for EL emissions from sources with <97% CH4. This assumption results in a reasonable constraint on the potential magnitude of BHFS-induced error unless EL emissions from low %CH4 sources were higher than EL emissions from high %CH4 sources. Here we discuss three reasons why our assumption is a more likely interpretation of the limited data available. First, the mass flow rate from a leaking source is a function of upstream pressure and orifice size (Green and Perry, 2007). We are unaware of a technical basis for why natural gas production sites with low %CH4 should have higher upstream pressures or physically larger leak points than high %CH4 sites. While the EL dataset in Allen et al. (2013b) includes site-level wellhead and separator pressure data for 80% and 60% of sites, respectively, analysis is inconclusive because it is not possible to determine what pressure corresponds to specific ELs. Notwithstanding, the median wellhead pressures at sites >97% CH4 and <97% CH4 are roughly equivalent (∼400 psi), while the median separator pressure is higher at sites with intermediate CH4 content (91–97%) than at sites with < 91% or >97% CH4 (610, 170, and 310 psi, respectively). There is no significant correlation between the magnitude of emissions and pressure (neither wellhead nor separator pressures), likely due to the uncertainty of attributing a specific pressure to each leak. Supplemental Material Spreadsheet S1 contains the pressure data and calculations discussed above.

Second, individual ELs with equal whole gas emission rates would have higher CH4 emission rates at high %CH4 sites compared to low %CH4 sites, because a larger fraction of the gas emitted is CH4. As a result, the effect of gas composition on the magnitude of CH4 emissions supports the conservative nature of our second bounding assumption. We note that the magnitude of this effect (<20–30%) is insufficient to explain the much larger observed difference between EL emissions at low and high %CH4 sites (a factor of 2 to 4, see Supplemental Material Spreadsheet S1).

Third, we considered the fact that low %CH4 sites often co-produce hydrocarbon liquids (oil or condensate). Production of hydrocarbon liquids would require additional, and/or potentially more complex, equipment such as separators, tanks, compressors and liquid-level pneumatic controllers than are present at high %CH4 sites that produce no hydrocarbon liquids. The larger population of potentially leaking components may result in a larger number of leaks per well: in Allen et al. (2013a), sites with <91% CH4 had slightly more leaks per well compared to sites with >91% CH4 (0.67 vs. 0.54 leaks/well). It is also possible that the use of an infrared camera to identify leaks can contribute to the higher rate at which leaks are identified at lower %CH4 sites since smaller leaks can be detected more readily due to the infrared camera’s enhanced sensitivity to non-CH4 hydrocarbons (Benson et al., 2006). However, even though low %CH4 sites may have more leaks than high %CH4 sites, there is no known relationship between the number of ELs at individual sites and the amount of gas emitted by individual ELs. In sum, we find these three potential factors are not likely to invalidate our assumption that the mean of ELs at high %CH4 sites should be larger than the mean at low %CH4 sites, and find no other evidence to the contrary. We thus conclude it is reasonable to assume the subset of ELs believed to be unaffected by BHFS failure (high %CH4 sites) represents a plausible basis for constraining the potential error in BHFS measurements made at low %CH4 sites.

A final assumption used to develop our constrained estimate of possible error due to BHFS sensor failure in the UT Production Studies is that the same scaling factors we determined for ELs also apply to the CIP dataset. Specifically, we applied an increase of 40–80% to CIP results, even though they do not exhibit patterns consistent with BHFS malfunction (CIP results are presented in the Supplemental Material Spreadsheet S1). The dataset for CIPs was smaller (N = 62) than for ELs, with a narrower range of %CH4, with relatively fewer measurements that could have been affected. Only three sources were <91% CH4 and 20 were <95% CH4; no sources were >97% CH4. The mean of measurements <95% CH4 was more than twice as high as the mean of measurements >95% CH4. Therefore, the direction of the bias for CIPs is the opposite of what would be expected if sensor transition failure had substantially affected results for sources with low %CH4. It is also noteworthy that seven CIP measurements for sources between 92–96% CH4 were higher than 0.4 scfm CH4, the volumetric emission threshold that Howard (2015b) estimates to represent the threshold for sensor transition failure. Thus, the CIP dataset suggests that sensor transition failure does not always occur for measurements in the 91–97% CH4 range, as we assume in the bounding scenario. In consideration of these factors, our constrained estimate is likely to overestimate the potential effect of BHFS malfunction on CIP measurements.

This work does not address Howard’s (2015b) analysis of results presented in Allen et al. (2013a) that compare site-level tracer flux measurements relative to component-by-component emission aggregations at sampled sites. While Howard (2015b) attributes higher estimates based on the tracer flux method to BHFS failure, we find that analysis to be inconclusive due to limitations in the data. Most notably, BHFS measurements were the basis for less than half of the on-site emissions estimate at two-thirds of sites; the balance of emissions was based on estimation methods (engineering estimates or emission factors). Therefore, any discrepancy may be due to an underestimate of modeled emissions from tanks or other unmeasured sources for which recent data suggest this might be the case (Lyon et al., 2016), rather than BHFS failure.


Potential BHFS sensor transition failures in the UT Production Studies would not affect our fundamental understanding that CH4 emissions from the natural gas supply chain are substantial or that production site emissions, specifically, are an important contributor to a larger problem that needs to be addressed. For example, the estimated CH4 leakage of Barnett Shale natural gas (where 1.6% of gas produced and delivered in that region is emitted) increases the 20-year cumulative radiative forcing of the natural gas fuel-cycle CH4 and CO2 emissions in the region, compared to CO2 emissions from complete combustion of each unit of produced natural gas, by 50% (Zavala-Araiza et al., 2015b).

The possibility that the BHFS may under predict emissions from natural gas sources deserves careful examination. Given the widespread use of the BHFS in the oil and gas industry, this paper is not intended to diminish potential implications of inaccurate emission measurements from an individual source. Systematic BHFS malfunction in regions where natural gas contains low CH4 could lead to underestimation of regional leakage estimates for source-specific populations. For example, if BHFS malfunction in the UT Production Studies had been the sole cause for differing whole gas emission rates between EL emissions in the low-%CH4 Rocky Mountain sites and high-%CH4 Appalachian sites, then reported Rocky Mountain leakage would have been underestimated by a factor of 4 (Supplemental Material Spreadsheet S1). BHFS users should be provided guidance on ways to prevent sensor transition failure when using the device. Howard et al. (2015) indicate that daily calibration and updated versions of firmware may minimize the occurrence of sensor transition failure.

At the same time, it is important that the effect of possible point source measurement errors on the results of previously published work be placed in context, particularly with regards to regional and national studies on CH4 emissions from oil and gas systems. Measurement error is not the only challenge that researchers face when scaling measurement data from a sampled subset of a larger population. The challenge is compounded when the sampled population is heterogeneous along many dimensions, as is the case for natural gas production sites. Factors such as skewness of emission distributions, gas composition, well age, gas and liquids production, operating practices, and regulatory requirements, to name a few, complicate the collection of a representative sample.

For some emission categories, the effect of uncertainties in activity data may be an equally or more important source of uncertainty than measurement uncertainty in scaling up measurements into a national emission estimate. For example, Allen et al. (2015a) observed a nearly three-fold higher count of pneumatic controllers per well than estimated in the 2014 EPA Greenhouse Gas Inventory (GHGI) (2.7 vs. 1.0 controllers per well, respectively). The GHGI pneumatic count is also lower than what natural gas producers reported for 2014 to the Greenhouse Gas Reporting Program (1.6 controllers per well) (U.S. Environmental Protection Agency, 2015). Another example is ELs, for which Allen et al. (2013a) report an emission factor per well (total measured leak emissions divided by total wells at sites sampled). Yet, because of variability in the number of wells and equipment located on natural gas production sites, there is high uncertainty in applying a per-well emission factor nationally. If the national well count per site differed significantly from the count in Allen et al. (2013a), then estimating a national total for EL emissions based on the national well count alone would produce a biased result. We are unaware of any comprehensive, national-scale estimate of well counts per site. However, a study (Lyon et al., 2016) of seven US producing basins that used a spatial buffering technique to join proximate wells into sites reports an average of 1.4 wells per site (range of 1.1–2.3) – less than half the value observed in the Allen et al. (2013b) dataset. If the count of wells per site in Lyon et al. (2016) is representative for all national well sites, then the bias introduced by activity data when scaling EL emissions would be larger than this work’s constrained estimate of the potential underestimation in the national emission factor for ELs due to BHFS malfunction measurements reported in Allen et al. (2013a).

Howard et al. (2015) suggest sensor transition failure in the BHFS may play a role in the discrepancy between top-down estimates and bottom-up measurements, because historical inventories rely on measurements made with the BHFS. However, emerging data suggest that emission underestimation due to BHFS malfunction would only explain a small part of the difference, largely because the BHFS is unsuitable for measuring emissions from the sources that are likely to be the main cause of the discrepancy. Recent work documenting agreement between top-down and bottom-up estimates for oil and gas CH4 emissions in the Barnett Shale highlights the importance of improved activity data (gathering facility counts in particular) and effective accounting for the influence of high-emitting facilities (Zavala-Araiza et al., 2015b). For example, the highest emitting 10% of production sites in the Barnett accounted for over 80% of all production site emissions; in turn, production sites were responsible for more than half of all oil and gas emissions. Understanding the component-level drivers of high-emitting facilities is a critical next step. Initial work indicates that abnormal or otherwise avoidable operating conditions, including malfunctions, contribute a substantial fraction of emissions at the highest emitting sites (sometimes called “super-emitters”) (Zavala-Araiza et al., 2015a; Mitchell et al., 2015, Zavala-Araiza et al., n.d.). A helicopter-based infrared camera survey of over 8,000 well pads in seven U.S. oil and gas production basins found roughly 500 high-emitting sources with hydrocarbon emission rates over 3.1 scfm; over 90% of observations were from tanks, with most of the remainder from flares, oil trucks, and dehydrator vents (Lyon et al., 2016). None of these source types were among the five assessed in the UT Production Studies. Importantly, the emission rates from the highest emitting sources exceed the measurement range of the BHFS (maximum emissions of 6–8 scfm) (Bacharach, Inc., 2015). Additionally, the BHFS is ill-suited for tank emission measurements due to their high concentrations of non-CH4 hydrocarbons and airflow constraints (Brantley et al., 2015). Accurately quantifying all emissions from natural gas production sites will likely require improved measurement approaches such as site-level, downwind measurements to fully capture large emission sources that cannot be measured with the BHFS (Subramanian et al., 2015; Roscioli et al., 2015; Brantley et al., 2014; Rella et al., 2015; Yacovitch et al., 2015; Lan et al., 2015; Omara et al., 2016).


© 2016 Alvarez et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Supplemental material

Text S1.
Additional analysis examining the possible effect of gas production rates on emissions.
File Type: DOC
File Size: 0.02 MB

Spreadsheet S1.
Excel file with the EL and CIP emission measurement data reported Allen et al. (2013a); their reported site pressure and production data; calculations presented in this work; and additional tables and figures referenced in the main text.
File Type: XLS
File Size: 0.26 MB

Table S1.
Mean and median absolute emission rates (standard cubic feet per minute; scfm) and proportional loss rates (% of site-produced CH4) of EL subsets with site gas composition greater or less than 91% and 97% CH4.
File Type: DOC
File Size: 0.02 MB

Figure S1.
Proportional loss rates of individual equipment leaks as a function of site %CH4.
File Type: DOC
File Size: 0.05 MB

Figure S2.
Cumulative distribution functions of proportional loss rates of individual equipment leaks (EL) in Allen et al. (2013a).
File Type: DOC
File Size: 0.04 MB

Figure S3.
Histograms of EL proportional loss rates at low % CH4 vs. high % CH4 sites (paired comparisons are stacked vertically): A. >91% CH4; B. <91% CH4; C. >97% CH4; D. <97% CH4.
File Type: DOC
File Size: 0.03 MB


Contributed to conception and design: RAA, DRL, SPH

Contributed to analysis and interpretation of data: RAA, DRL, AJM, ALR

Drafted and/or revised the article: RAA, DRL, AJM, ALR, SPH

Approved the submitted version of the article: RAA, DRL, AJM, ALR, SPH

Competing interests

The authors declare no competing financial interest. Three co-authors work for Environmental Defense Fund (EDF), a non-profit organization funded by philanthropic contributions, which does not accept donations from the oil/gas industry in accordance with its corporate donations policy ( EDF was a sponsor of the University of Texas (UT) Production Studies. The other co-authors were not involved with the UT Production Studies; they were lead investigators in separate projects focused on quantifying methane emissions from the U.S. natural gas supply chain, sponsored in part by EDF.


The authors thank five anonymous reviewers for helpful suggestions that improved the manuscript.


  1. Allen DT , Pacsi AP , Sullivan DW , Zavala-Araiza D , Harrison M , et al. 2015a. Methane emissions from process equipment at natural gas production sites in the United States: Pneumatic controllers. Environ Sci Technol 49: 633–640.

  2. Allen DT, Sullivan DW, Harrison M. 2015c. Response to comment on “Methane emissions from process equipment at natural gas production sites in the United States: pneumatic controllers. ” Environ Sci Technol 49: 3983–3984.

  3. Allen DT , Sullivan DW , Zavala-Araiza D , Pacsi AP , Harrison M , et al. 2015b. Methane emissions from process equipment at natural gas production sites in the United States: Liquid unloadings. Environ Sci Technol 49: 641–648.

  4. Allen DT , Torres VM , Thomas J , Sullivan DW , Harrison M , et al. 2013a. Measurements of methane emissions at natural gas production sites in the United States. P Natl Acad Sci USA 110: 17768–17773.

  5. Allen DT, Torres VM, Thomas J, Sullivan DW, Harrison M, et al. 2013b. “Sources Dataset” Electronic Data Set. Measurements of methane emissions at natural gas production sites in the United States. P Natl Acad Sci USA 110: 17768–17773 .

  6. Bacharach, Inc. 2015. “Hi-Flow Sampler™” for natural gas leak rate measurement, Instruction Manual, Rev. 7 -- July 2015. Accessed February 2, 2016.

  7. Benson R, Madding R, Lucier R, Lyons J, Czerepuszko P. 2006. Standoff passive optical leak detection of volatile organic compounds using a cooled InSb based infrared imager. AWMA 99th Annual Meeting Papers Vol. 131 .

  8. Brandt AR , Heath GA , Kort EA , O’Sullivan F , Pétron G , et al. 2014. Methane leaks from North American natural gas systems. Science 343: 733–735.

  9. Brantley HL, Thoma ED, Eisele AP. 2015. Assessment of volatile organic compound and hazardous air pollutant emissions from oil and natural gas well pads using mobile remote and on-site direct measurements. J Air Waste Manage 65: 1072–1082.

  10. Brantley HL, Thoma ED, Squier WC, Guven BB, Lyon D. 2014. Assessment of methane emissions from oil and gas production pads using mobile measurements. Environ Sci Technol 48: 14508–14515.

  11. Green DW, Perry RH. 2007. Perry’s Chemical Engineers’ Handbook, Eighth Edition . New York: McGraw-Hill.

  12. Howard T, Ferrara TW, Townsend-Small A. 2015. Sensor transition failure in the high volume sampler: Implications for methane emissions estimates from natural gas infrastructure. J Air Waste Manage 65: 856–862.

  13. Howard T. 2015a. Comment on “Methane emissions from process equipment at natural gas production sites in the United States: Pneumatic controllers”. Environ Sci Technol 49: 3981–3982.

  14. Howard T. 2015b. University of Texas study underestimates national methane emissions at natural gas production sites due to instrument sensor failure. Energy Sci Eng 3: 443–455.

  15. Kirchgessner DA, Lott RA, Cowgill RM, Harrison MR, Shires TM. 1997. Estimate of methane emissions from the U.S. natural gas industry. Chemosphere 35: 1365–1390.

  16. Lan X, Talbot R, Laine P, Torres A. 2015. Characterizing fugitive methane emissions in the Barnett Shale area using a mobile laboratory. Environ Sci Technol 49: 8139–8146.

  17. Lyon DR , Alvarez RA , Zavala-Araiza D , Brandt AR , Jackson RB , et al. 2016. Aerial surveys of elevated hydrocarbon emissions from oil and gas production sites. Environ Sci Technol 50: 4877–4886.

  18. Mitchell AL , Tkacik DS , Roscioli JR , Herndon SC , Yacovitch TI , et al. 2015. Measurements of methane emissions from natural gas gathering facilities and processing plants: Measurement results. Environ Sci Technol 49: 3219–3227.

  19. Modrak MT , Amin MS , Ibanez J , Lehmann C , Harris B , et al. 2012. Understanding direct emission measurement approaches for upstream oil and gas production operations, control # 2012-A-411-AWMA. In Proceedings of the 105th Annual Conference of the Air & Waste Management Association, San Antonio, Texas, June 19–22, 2012 . San Antonio, TX: Air & Waste Management Association.

  20. Moore CW, Zielinska B, Pétron G, Jackson RB. 2014. Air impacts of increased natural gas acquisition, processing, and use: A critical review. Environ Sci Technol 48: 8349–8359.

  21. Omara M , Sullivan MR , Li X , Subramanian R , Robinson AL , et al. 2016. Methane emissions from conventional and unconventional natural gas production sites in the Marcellus Shale Basin. Environ Sci Technol 50: 2099–2107.

  22. Rella CW, Tsai TR, Botkin CG, Crosson ER, Steele D. 2015. Measuring emissions from oil and natural gas well pads using the mobile flux plane technique. Environ Sci Technol 49: 4742–4748.

  23. Roscioli JR , Yacovitch TI , Floerchinger C , Mitchell AL , Tkacik DS , et al. 2015. Measurements of methane emissions from natural gas gathering facilities and processing plants: Measurement methods. Atmos Meas Tech 8: 2017–2035.

  24. Subramanian R , Williams LL , Vaughn TL , Zimmerle D , Roscioli JR , et al. 2015. Methane emissions from natural gas compressor stations in the transmission and storage sector: Measurements and comparisons with the EPA Greenhouse Gas Reporting Program protocol. Environ Sci Technol 49: 3252–3261.

  25. U.S. Environmental Protection Agency. 2014. Inventory of U.S. Greenhouse Gas Emissions and Sinks: 1990–2012. EPA 430-R-14-003 .

  26. U.S. Environmental Protection Agency. 2015. GHGRP Petroleum and Natural Gas Systems Sector Industrial Profile 2011–2014.

  27. Yacovitch TI , Herndon SC , Pétron G , Kofler J , Lyon D , et al. 2015. Mobile laboratory observations of methane emissions in the Barnett Shale region. Environ Sci Technol 49: 7889–7895.

  28. Zavala-Araiza D , Alvarez RA , Lyon DR , Allen DT , Marchese AJ , et al. n.d. Super-emitters in natural gas infrastructure are caused by abnormal process conditions. Nature Communications : accepted for publication.

  29. Zavala-Araiza D , Lyon D , Alvarez RA , Palacios V , Harriss R , et al. 2015a. Toward a functional definition of methane super-emitters: Application to natural gas production sites. Environ Sci Technol 49: 8167–8174.

  30. Zavala-Araiza D , Lyon DR , Alvarez RA , Davis KJ , Harriss R , et al. 2015b. Reconciling divergent estimates of oil and gas methane emissions. P Natl Acad Sci USA 112: 15597–15602.

comments powered by Disqus