ET25SWE0038 - Optimization of Deemed Data Collection Requirements

Deemed measures rely on competing goals of accuracy and volume to ensure that the gross claims are reasonable. In the current California eTRM (electronic Technical Reference Manual), deemed measures increasingly have relied on more extensive and more complex data collection to ensure a claim correctly aligns to the intent of a single permutation. Even though the eTRM and deemed measure packages are built with an intended accuracy of 10%, additional permutations are often added that create differences smaller than 10%. 

 However, an inverse relationship exists between these two goals since the strategy of making a single claim more accurate requires more documentation; more documentation increases the cost of supporting each customer claim; and higher customer acquisition costs will decrease the volume of claims. In some case, increased customer acquisition costs outweigh the benefits of a deemed measure, which could add to the reasons for why the measure goes unused. This project proposes to examine the benefits/tradeoffs of streamlining the data collection requirements to minimize customer acquisition cost while still maintaining reasonable accuracy in the claimed savings. 

 This project proposes to examine a group of 5 commonly implemented measures and a group of 2 unused measures from the CA portfolio in 2023-2024. The project team, led by TRC, will work with implementers and measure developers to understand the measure input parameters (i.e., product specs and operational characteristics) that drive measure package results (ie, kWh, kW, therms) and cost-effectiveness outputs such as Total Resource Cost (TRC) and Total System Benefit (TSB) in these measure packages. In addition, the team will work with measure developers and Program Administrators (PAs) to document the sensitivity of measure parameters on results and outputs. Finally, the team will work with implementers to establish estimates for the cost of data collection for these measure input parameters. Outcomes include clear data collection recommendations for each measure package with the corresponding expected uncertainty and a framework for how the methods from this project can be extended to additional measure packages. 

 By driving down data collection cost, ideally the volume of claims can increase significantly. With increased volume, the assumptions made in deemed claims will become more reliable estimates of the population, making gross savings claims more accurate. In some cases, the customer acquisition cost could still be greater than the Total System Benefit, which can lead to important next steps in modifying a measure package to accept less accuracy or in taking the controversial but important step of sunsetting the measure package when issues cannot be resolved. 

Deemed measures rely on the competing goals of accuracy and volume to ensure that gross claims are reasonable. In the current California electronic Technical Reference Manual portfolio, this balance is not always well aligned; as of PY2025, more than 40 measure packages, representing over 25 percent of the portfolio, have not been part of any savings claims since PY2022. This project addressed the need to better align data collection requirements with the measure parameters that materially affect total system benefit, total resource cost, and claim integrity. 

The project developed and piloted a framework for evaluating data collection burden across a representative group of five commonly implemented measure packages and two unused or rarely used measure packages in the California portfolio from 2023 to 2025. The approach combined electronic Technical Reference Manual permutation outputs, recent claims trends, stakeholder feedback, sensitivity analysis, and structured estimates of data collection cost by format and delivery type. This allowed the project team to compare the value of additional documentation against the benefits it protects. 

The results show that several medium and high effort documentation requirements are not tied to parameters that materially drive total system benefit or total resource cost. In some cases, data collection cost approaches or exceeds total system benefit, reducing implementer value and discouraging participation. The analysis also found that photo-heavy and site-access-dependent requirements can become disproportionately burdensome at scale, especially for midstream, upstream, or low-rigor delivery types. 

This project demonstrates that data collection requirements should be set deliberately before measure packages are introduced or revised. Requirements tied to sensitive parameters should be retained and automated where possible, while low sensitivity requirements should be simplified, removed, or replaced with clearer standard text to preserve enforceability without suppressing claim volume.