Advancing the Art of PRISM Analysis
	by Margaret F. Fels Kelly Kissock Michelle A. Marean and Cathy Reynolds
	How do you know how much energy a conservation program has actually saved? A new version of the computer program PRISM now makes it easier to transform run-of-the-mill billing data into statistically sound estimates of energy savings.

Since the original version of PRISM® (PRInceton Scorekeeping Method, developed and copyrighted by Princeton University) was released in 1986, it has been used by utilities, private companies, government agencies, and universities to estimate energy savings from conservation programs. PRISM (Advanced Version 1) produces even more reliable savings estimates and expanded statistical capabilities from the same readily available utility billing data. In addition, the new PRISM has a Windows-based interface to make it much easier to learn and more user-friendly.

PRISM is a statistical procedure that uses a year of monthly billing data from a house or building to produce a weather-adjusted index of energy consumption. The index is called Normalized Annual Consumption, or NAC (see "PRISM Parameters"). The difference between the NACs in pre- and postweatherization periods equals the total energy savings. PRISM can produce analyses for large samples of houses or buildings participating in a program, as well as for control groups.

PRISM requires only two sets of data: utility bills (including meter-reading dates and energy use computed from the meter readings) from before and after the installation of a conservation measure; and average daily temperatures from a nearby weather station (see Figure 1). The same files that you may have lying around from old PRISM runs will still work. PRISM gives results in terms of base-level versus heating consumption and base-level versus cooling consumption, and the building's reference temperature for heating or cooling. (The heating reference temperature is the average outside temperature at which a house's heating system kicks on; the computer program chooses the reference temperature that best matches the data.) Statistical measures of reliability are generated to allow the evaluator to decide how much confidence to place in NAC, savings, and the other PRISM parameters (see also "PRISM: A Tool for Tracking Retrofit Savings," HE Nov/Dec '87, p. 27, and "Now That I've Run PRISM, What Do I Do with the Results?" HE Sept/Oct '90, p. 27). Here ends the similarity between the old and new PRISM.

The Advanced PRISM project at Princeton University was funded by the Electric Power Research Institute, the Wisconsin Center for Demand-Side Research, and eight utilities, and it focused on "model tuning and data pruning." Model tuning meant adding such features as a Heating-and-Cooling model, a Robust model, and an Aggregate version, while data pruning included improving the reliability and usefulness of the data with functions like automated data correction and "outlier" detection. The objective was to improve the PRISM methodology in order to make the best possible use of the information available in billing data.

All of the elements of PRISM are combined into a standardized protocol for PRISM analysis, producing a more reliable set of estimates from a PRISM run on one year (or more) of data for one house or building. Once PRISM has been run on a year of pre-retrofit and post-retrofit data, weather-adjusted savings (and associated reliability statistics) for each building, as well as group savings estimates for all buildings in the participant and nonparticipant (control) groups, may be calculated. The Advanced Version of PRISM computes the individual-building savings and also provides statistical and graphical comparisons of the savings in the different groups. Reliability criteria for computing savings may be applied to isolate the subset of buildings that have reliable results. From the results, a one-page savings summary and summary distribution graphs (histograms) are produced. These can be used for standardized PRISM reporting to state or federal regulators.

The old PRISM could evaluate only heating or cooling separately for each fuel. The Heating-Only (HO) model has worked well for heating fuels (for example, if natural gas is used for heating and other purposes, or if electricity is used for heating, lighting, and appliances other than cooling), while the Cooling-Only (CO) model has also worked well when electricity is used for air conditioning but not for heating. But if a house used electricity for both heating and cooling, the user was out of luck, because the program could not accurately determine base consumption versus heating or cooling.

In the new PRISM, a Heating-and-Cooling (HC) model combines heating and cooling in the analysis of buildings that use the same fuel for both. Since it uses information about daily temperatures, the program can even account for possible overlaps, when both heating and cooling may be used in the same month. (The separate heating and cooling models are basically the same as in the old PRISM for houses that use a single fuel for each.)

A sophisticated feature of new PRISM is its ability to do a savings analysis on a set of buildings, not just on individual houses. The user has the option to define reliability criteria for buildings to be included in the savings summary. Savings results (both total energy savings and percentages) are summarized in graphs and tables. If the user has organized the meter file for "pre" and "post" savings analysis, and for control versus participant savings, the complete savings summaries are easily produced.

The savings summary includes

• Number of sites in the sample (includes participant and nonparticipants in the conservation program).
• Median and mean savings in kWh per year for each group.
• Percentage of savings for each group and control-adjusted savings for participant group.
• Model types used (HO, CO, HC).
• Median R² and CV(NAC) values.

Utilities and others can also use the Advanced Version of PRISM to determine trends in energy use within large categories of customers by using an Aggregate version of the HO, CO, or HC model. The Aggregate version doesn't look at data points for individual houses; instead it does weather adjustment for overall utility sales data by customer class over many years. PRISM can use utility records of total sales by rate category and divide by the number of customers to get an average customer usage. It takes into account the fact that meters are read at different times during the month, and the reliability statistic (R²) tends to be very high.

The new PRISM has several statistical improvements that make the results more reliable and easier to interpret. It can now detect and correct estimated readings and meter-reading errors; test for flat (non-weather-dependent) consumption, in terms of a Flatness Index (FI); detect outliers in the consumption data and use the Robust version if appropriate; and automatically determine, for each building, the appropriate model for each period of analysis.

In addition to the original (Regular) models, Robust versions of the HO and CO models have been added. After a house's data are entered, the program maps energy use per day against heating (or cooling) degree-days per day and draws a representative line that all the points fall near or on. If there is a point that does not seem to fit in with the trend represented by the other points (perhaps because of a data entry mistake), it is an "outlier."

In the Regular model, all data points (each representing a meter reading) have an equal influence on the analysis, even those that should be considered outliers. The Robust model, on the other hand, would give each of the 11 conforming data points more weight than the outlier. It gleans as much information from the data point as it can, without letting it interfere disproportionately with the fit.

The new PRISM can detect an outlier in the Regular model and recommend that the user rerun the house with the Robust model. The user may also choose to run all houses in the Robust mode.

The Flatness Index (FI) indicates how variable the consumption is for any year of data. A house with no heating or cooling with the fuel being examined will most likely have a low FI, meaning that the usage didn't increase or decrease much on a seasonal basis. Likewise, a house with both heating and cooling will have a high FI because the usage varies a lot.

PRISM uses a reliability screen to remove unreliable houses from the sample. The flatness index allows it to use data that might otherwise be discarded as unreliable. For instance, if a house has a low reliability (R²) but also a low FI, PRISM can still get a good savings estimate from the Normalized Annual Consumption. Thus one can use PRISM intelligently for houses that don't have much heating and cooling consumption.

As with the old PRISM, known estimated readings should be combined in the initial preparation of the meter file before running the new PRISM. Since an estimated reading one month is generally compensated the following month by an actual reading, the remedy is to combine the consumption for the two periods on either side of the estimated reading. Otherwise, the reliability of the results may be greatly reduced. In the event of a missed estimated reading or a meter-reading error, new PRISM now offers an estimated-reading detector, with automated correction as needed.

The Check for Estimated Readings feature can be applied to a data set to find any houses with probable estimated readings. PRISM then looks for a high/low pair, which occurs, for example, when a meter is read too high one month, making the consumption appear high. The next month the meter is read correctly and consumption appears low. This shows up in the PRISM consumption plot as one data point well above and one well below the line. PRISM can correct the meter file by combining the two data points, with the combined data point falling much nearer to the line.

This simple data correction improves the quality of results enormously, increasing R² and decreasing CV(NAC). Thus PRISM is able to convert many runs from "unreliable" to "reliable," thereby increasing the percentage of reliable cases in a savings analysis.

Although information on the type of HVAC system can often indicate whether the HO, CO or HC model should be used, such information (if available) can be incorrect, or inconsistent with the heating and cooling signals seen in the consumption data. PRISM's new Automated Model Selection feature, which is one of the options that the user may select on the model selection screen, uses the winter and summer patterns in the consumption data to determine whether the HO, CO, or HC model seems most appropriate. The initial model selection is then verified, or revised, based on an assessment of the reliability of the results.

To make PRISM more useful and accessible and to accommodate all of the new features, a user interface was developed in an interactive Windows-based program. Users can click on commands with a mouse and use pull-down menus to select new and old PRISM options. In contrast to the old PRISM, the new version is also practically self-teaching.

For demonstration purposes, we took a sample of 22 houses from a residential weatherization program conducted in 1984 in Washington State, in which the utility reimbursed up to 85% of measure installation costs after a walk-through audit. (This small sample is not intended to represent actual savings for the program.) We ran through the following steps of the recommended PRISM Protocol:

1. Choose model by running automated model selection (with verification) or, if HC is not a possible model choice, by running HO or CO on entire meter file.

2. Check for estimated readings (or meter-reading errors) and rerun PRISM on the corrected meter file.

3. Check for outliers and run the Robust version of the model on all cases with outliers.

From the meter file containing the 22 houses, PRISM produced the savings distribution graphs shown in Figure 2.

Although PRISM initially chose the HC model for seven of the cases in the data set, the HO model, after verification, was the final choice for all of them. PRISM's data-pruning processes caught data errors and other quirks that gave an initial indication of both heating and cooling energy use for these customers. Estimated readings were found and corrected in two cases. Outliers were identified in five other cases, and use of the Robust version of the HO model improved the results. Three of them had weak cooling signals, and for such cases either the HC or the HO model was a good choice.

The median savings found for the participant group was 17%, versus 4% in the comparison (non-participant) group, for a net savings of 13%.

Figure 2. Savings distribution graphs for a sample of houses used to evaluate a utility DSM program. The graph on the left shows savings in the non-participant group. Savings for the houses that participated in the program are shown on the right.

Looking at some of the houses in the sample will illustrate more specifically how PRISM works. The starting point for a new PRISM analysis is the same as before: a meter file of monthly consumption data (see Table 1), and a temperature file of daily temperature data from a nearby weather station. But whereas the old version ran individual houses and gave only individual house results, PRISM now takes the user all the way from the raw billing data to summaries of distributions of savings.

Running the default (HO) model on the billing data, using Seattle temperature data, we can get a picture of this house's savings by comparing the plots of raw consumption data for the pre- and post-retrofit periods (see Figure 3). We can look at the reliability indices to see how much faith to put into the results. In this case, the HO model works very well on both periods, with R² > 0.9 and CV(NAC) < 5% in both cases. (This easily passes the recommended reliability criteria for "good" PRISM models: R² > 0.7 and CV(NAC) < 7%.) The main change that the user will observe in the PRISM results is a decline in the reference temperature, possibly due to installation of setback thermostats and customer education. Overall, the NAC declined by 28%, from 12,600 kWh/year to 9,100 kWh/ year.

Figure 3. Consumption plot for a sample house, before (a) and after (b) weatherization.

To illustrate the new HC model, we chose a house from a data set in Houston. Once the meter and temperature files are loaded, the user may select the HC model from the model selection screen. The consumption plots for this house show clearly a heating signal in winter and a cooling signal in summer (see Figure 4). The resulting R² of 0.93 and CV(NAC) of 3.4% indicate that the HC PRISM model works well in this case.

Figure 4. Consumption plots for a house that uses electricity for both heating and cooling.

The new PRISM offers a one-page standardized summary of the median- and mean-savings statistics, with a record of the reliability criteria that were selected, the models that were used, and the average reliability statistics (R² and CV[NAC]) for each group. With this record, savings results should be reproducible, and easily compared across different programs.

PRISM (Advanced Version 1) is available to members of Electric Power Research Institute (EPRI) through the EPRI Software Center. Others can obtain copies from the Center for Energy and Environmental Studies, Engineering Quadrangle, Princeton University, Princeton, NJ 08544. Tel: (609) 258-4774. The software comes with sample data files and a detailed users' guide that includes tutorials and an indexed reference manual. The cost of the program is $795 for utilities, government agencies, and energy consulting and energy service companies. The price for colleges, universities, and CAP agencies (and for additional site licenses) is $395.

For more background on the PRISM methodology, see the special Scorekeeping Issue of Energy and Buildings 9, no 1-2 (1986), which contains 16 papers on the methodology and its applications.

The derivation and validation of PRISM's Automated Model Selection algorithms are described in M. Fels, K. Kissock, and M. Marean, "Model Selection Guidelines for PRISM" (Or: "Now that HC PRISM Is Coming, How Will I Know When to Use It?") See Proceedings of the ACEEE 1994 Summer Study on Energy Efficiency in Buildings, 8.49-8.61. ACEEE, Washington, DC, 1994).

Margaret F. Fels is senior research scientist, Kelly Kissock is research associate, Michelle A. Marean is research assistant, and Cathy Reynolds is senior research assistant with the Center for Energy and Environmental Studies at Princeton University.