Better Buildings Neighborhood Program
DOE's Better Buildings Neighborhood Program (BBNP) was funded through the American Recovery and Reinvestment Act (ARRA) of 2009. Initially, DOE made 25 awards to local governmental or nonprofit organizations in amounts ranging from $1.2 million to $40 million. Nine similar grantees from DOE's Formula Energy Efficiency and Conservation Block Grant (EECBG) program and seven from DOE's State Energy Program (SEP) solicitation resulted in 41 Better Building grantees, with award allocations of $508,302,786 in funding for BBNP projects. This article summarizes a preliminary process and market effects evaluation of the BBNP that was conducted by a team led by Research Into Action and managed by Lawrence Berkeley National Laboratory (LBNL). I was the project manager. The BBNP has three objectives:
- to initiate building energy upgrade programs that promote projects estimated to achieve energy savings in more than 40 communities;
- to demonstrate more than one sustainable business model for providing energy upgrades to a large percentage of the residential and/or commercial buildings in a specific community; and
- to identify and promote the most effective approaches to completing building energy upgrades that support the development of a robust retrofit industry in the United States.
As part of the BBNP, DOE focuses on four pillars needed to support an effective energy upgrade program. These pillars are marketing, financing, workforce, and data and reporting. Accordingly, the BBNP seeks to increase the overall energy efficiency of residential and nonresidential facilities through home and building assessments, through building up a trained workforce, and through financing and incentives to support energy efficiency upgrades.
The preliminary process and market effects evaluation is based upon what grantees say about the program approximately two years after the grants were awarded. The preliminary evaluation is based on data from many sources. These sources include program databases; U.S. census data; grantee web sites; the BBNP Google web site; and in-depth interviews with 35 grantees, 11 DOE staff, four DOE support contractor staff, 6 nongovernmental stakeholders, and 26 market informants. The evaluation team also conducted surveys with 189 contractors participating in the BBNP and 151 nonparticipating contractors, as well as 164 equipment suppliers serving the owners of homes and commercial buildings in 22 of the grantee locations.
For this preliminary evaluation, the evaluation team interviewed 35 of the 41 grantees. These grantees were selected after discussions with DOE's account managers, who worked with the grantees to identify their needs for support in designing, developing, and implementing their programs, and then helped the grantees to find that support. Account managers identified those grantees that, as of late spring 2012, had programs fully up and running, and that had not undergone recent changes in management or program design. The evaluation team also considered the number of audits or upgrades grantees had accomplished, the number of workers they had trained or certified, and their outlay on training as a percentage of total budget. The use of many criteria made it possible to include in the sample grantees that might not have qualified based on any one single criterion.
To identify factors contributing to grantee success, the evaluation team first identified the grantees that were most successful to date and then sought to determine what had made them successful. The evaluation team divided the sample into high, medium, and low success-to-date groups, each group comprising 10–12 grantees. The team next developed a composite metric of success to date, based on progress toward goal, rate of conversion of audits to upgrades, average cost per upgrade completed, and average cost per unit of energy saved. This metric was used for the EECBG grants only, since the SEP grants had not yet been awarded. The grantees with the ten highest success-to-date metric values were identified and were categorized as high-success grantees. Although the final evaluation process will explore alternative ways to measure success, the preliminary findings strongly suggest that our initial conclusions are robust and that they will probably remain valid when success is defined more precisely.
The evaluation team found many differences among grantees and among their programs. These included differences in type of organization, in prior experience with energy efficiency, in climate and type of building served, in services and measures offered, in the role played by private-sector firms in delivering program services, and in marketing methods. The factors most strongly correlated with the performance of the ten most successful grantees were
- working in partnership with financing organizations;
- working in partnership with nonprofit organizations; and
- having energy efficiency experience, either by having done energy upgrades in the community or by having collaborated with organizations that did these upgrades.
Partnerships with financing organizations are important. They make it easier for grantees to provide effective financing solutions. Nonprofits are effective because they are flexible and nimble, and thus able to adjust programs as needed; collaborating with them seems to make grantees more nimble as well. Finally, while the statistical association was not clear with respect to the energy efficiency experience metric, the interview data suggest that having strong energy efficiency experience in the community promotes community interest in upgrades, and having organizational experience provides the knowledge and capacity to develop an integrated and effective program.
Findings from other sources provide further support for the findings from grantee interviews. The evaluation team found preliminary evidence that market effects are emerging for grantee programs. This evidence is based on the surveys of participating and nonparticipating contractors and suppliers of energy-efficient equipment. Both participating and nonparticipating contractors agreed that the BBNP grantee programs were having a positive effect on their businesses and on the marketplace in general. The surveys also found that contractors believe there are more trained contractors than there were several years ago and that more contractors are marketing energy efficiency. The surveys found that suppliers believe there are more high-efficiency equipment and products on the market and that sales of these products are increasing. These growth rates are generally more pronounced in the areas with the most-successful grantees. However, it appears that spillover (upgrade activity among customers not participating in grantee programs) may be somewhat higher in the areas with the least-successful grantees, which have typically had less prior experience with energy efficiency.
The preliminary evaluation confirms that DOE's four pillars (marketing, financing, workforce, and data and reporting) are necessary components of an effective upgrade program. More importantly, this research confirms that these components must work together for an energy upgrade program to be successful. Further, the research confirms that there is no best way to implement each component and that each component must be integrated with the other three. Finally, the evaluation team found that it is necessary to gather the necessary data and reports from grantees to evaluate success and in order to obtain long-term funding.
Although the grantees and their programs vary widely, the evaluation team found that success was not associated with any specific type of organization, with climate or type of building served, with the specific services and measures offered, with the role played by private-sector firms in delivering program services, or with marketing methods. There is no "silver bullet" of activities that is likely to lead to success. Some activities have yielded good results, yet both successful and unsuccessful grantees engage in these activities, and both successful and unsuccessful grantees engage in some activities that have not been identified as promising.
At the close of the second year of the three-year grant period, nearly a quarter of the grantees have developed successful business models, and several of these models show promise of being sustainable beyond the funding period.
This article draws from a report prepared by an evaluation team led by Research Into Action for Lawrence Berkeley National Laboratory. Download the full report.
The evaluation team believes that the following key factors will increase the success of the BBNP, regardless of the variation among grantees and their programs. They recommend that in the final year of the program (2013), DOE should do the following:
- Encourage grantees to clearly identify who should have the role of selling the upgrade, and to provide sales training to those individuals.
- Encourage grantees to include messaging that emphasizes comfort and solutions to building problems, since such messaging appears to be influential.
- Encourage grantees in their continued efforts to simplify assessments and connect the assessment to the upgrade sales process.
- Encourage grantees to sponsor meetings that give contractors opportunities to share their experience and insights with each other and with the grantees' program teams.
- Encourage grantees to develop programs with components that drive demand and stimulate supply.
- Promulgate these components to market actors (for example, to retailers and distributors) who may be unfamiliar with the BBNP. This should encourage these market actors to support the program.
As I note above, this is a preliminary process and market effects evaluation. The evaluation team is working on a final process and market effects evaluation, and on a preliminary and a final-impact evaluation. These evaluations will be completed by the end of 2014, and the results will be distributed widely, including in a follow-up article in Home Energy.
Enter your comments in the box below:
(Please note that all comments are subject to review prior to posting.)
While we will do our best to monitor all comments and blog posts for accuracy and relevancy, Home Energy is not responsible for content posted by our readers or third parties. Home Energy reserves the right to edit or remove comments or blog posts that do not meet our community guidelines.