Appendix A: Overview of the CTPL

A.1. CTPL Overview.. 1

A.1.1. Summary. 1

A.1.2. Acknowledgements. 3

A.1.3. Protocol Definitions. 3

A.1.4. Literature Search. 4

A.1.5. Document and Protocol Review Matrix. 7

A.1.6. Commissioning Test Protocol Templates. 7

A.1.7. CTPL Information Database Tool8

 

A.1. CTPL Overview

A.1.1. Summary

The following section is an excerpts from “About this Product” in the CTPL Release 1.3. 

As part of a commissioning related market transformation program, Pacific Gas and Electric’s (PG&E) Customer Energy Management New Construction unit has undertaken a project to initiate the development of a library of commissioning test protocols.  PG&E believes that the building commissioning process as outlined in ASHRAE Guideline 1-1996, when properly applied to buildings, can result in buildings with superior systems, efficiency, and occupant comfort.  PG&E is interested in promoting the commissioning process among its customers and believes that a library of well-written, cost-effective verification check and functional test protocols that includes a template to produce new ones is needed by the commissioning industry. Our long-term objective is to help create a readily accessible library of verification check and functional test protocols archived in an updateable informational database.  For owners and providers new to commissioning, the library is intended to help reduce the amount of work required to fully and adequately commission a building and its systems, thereby streamlining the process.  For experienced owners and commissioning practitioners, the library will provide a state of the art source for procedures.  The library will also provide insight into the best practice currently available, aiding all parties interested in commissioning.

Our major focus has been to identify, in the current literature, component, equipment, and sub-system level protocols that could be integrated into an overall system specific test(s) tailored to a particular building, system or manufacturer.  We also identified gaps where they appeared to exist.  Project scope included identification of test protocols for lighting, including lighting controls and daylighting and HVAC systems, sub-systems and equipment including: boilers, furnaces & hydronic heating systems, chilled water plants, cool storage systems, commercial refrigeration, HVAC air handling units, heat pump systems, roof-top units, unitary fans, condensing units, and terminal units.  After completing the literature search and document review, this list has grown to include among others, energy management and control, service hot water, steam heating, wastewater treatment and electrical equipment.  

To aid in classifying protocols, we developed working definitions for what we concluded were the two basic classes of commissioning protocols: verification checks and functional tests.  See A.1.3 Protocol Definitions below for definitions.  These classes were then subdivided into sub-classes and types.  A literature search was then performed to identify both copyrighted and publicly available commissioning test protocols, which we then collected, cataloged using a detailed review matrix, and indexed in a database (Microsoft Access 97Ò). 

The review matrix provided a consistent method of identifying the key elements of the commissioning procedure under review.  Protocols that were practical and provided clear, meaningful instructions and acceptance criteria were preferred over those that were impractical or poorly written.  As a standard reference, a quality protocol was expected to include the test name, the conditions under which the test is to be performed, test duration, data to be gathered, method and location of measurements required, instrumentation and data acquisition requirements including measurement tolerance, results to be obtained including analysis calculations if required, specific measurable acceptance criteria and any notes of caution to the user.  Any special requirements that must be obtained or defined by the individual performing the test should also be listed along with a place to record actual test conditions, names of the test personnel, and results. 

Each review included an evaluation of protocol practicality and cost vs. benefit.  The evaluation required the reviewer to first score the likelihood of the material under review producing the desired result, and then rate each for complexity, skill required of the user, cost effectiveness, repeatability, as well as its relevance to the library being developed.  Procedures and protocols that provided the most potential for improved system performance, efficiency and reliability were scored higher than procedures and protocols that were technically viable but of less practical value in the real world, day-to-day operating environment.  See A.1.5 Document and Protocol Review Matrix below for more information on the review matrix. 

To date, the review team has evaluated 80 documents or accumulations thereof, and completed 523 individual protocol review summaries.  Allowing for multiple protocols per review, nearly 630 different protocols have been identified.  Utilizing the data gathered from these reviews, the project team developed a draft of a systems-based commissioning protocol template.  Eighteen separate template forms with detailed input fields and instruction sets were created for the various protocol classes, sub-classes and types.  Subsequent to the initial releases of the database library, a questionnaire was developed to poll users on potential improvements.  From this input, two new test procedures were developed; one for economizers and the other for DDC systems, to help refine the template forms and develop new ones as needed.  This revised protocol templates document, which now includes 20 forms, is provided with Developmental Release 1.3.  See A.1.6 Commissioning Test Protocol Templates for examples of these template forms.

The deliverables developed as part of this project have been assembled into an informational database product that includes four primary components: a document and protocol review database, a library of existing commissioning related protocols that we had permission to include, a user protocol archive and the protocol templates document. 

The document and protocol review database includes the document and protocol review tables, a document review form and a protocol review form, each allowing the user to enter new reviews, a document search engine, a protocol search engine, and an updateable review statistics table (Stats) that tracks the types of protocols identified.  Numerous enhancements to user functionality and instruction sets have been provided in Developmental Release 1.3.

The library of protocols provides a number of non-copyrighted commissioning related documents in Microsoft WordÓ rich text format that can be used as a resource for creating new protocols.  These protocols can be viewed if an electronic copy of the protocol in MS Word format is placed in the “/documents” subdirectory. 

A new user protocol section, added in Developmental Release 1.3, allows the user to archive any new commissioning protocols they might develop or acquire.  The protocol can be viewed if an electronic copy of the protocol in MS Word format is placed in the “/user_protocols” subdirectory.  The user can also browse through the input screens and view the index.   

The database Library is now available in Microsoft Access 97Ò and Access 2000Ò.  It is anticipated that it will be enhanced by and coordinate with the current CEC/PIER Functional Test Guide project being developed by LBNL and PECI.

A.1.2. Acknowledgements

Definition development – Thanks to Ted Cohen, past chair of ASHRAE GPC1-1989R, Karl Stum, Chad Dorgan, and Gerald Kettler, current members of ASHRAE GPC1-1996R, and Mike Kaplan, Pete Keithly, Gretchen Williams, and J. R. Andersen via the Building Commissioning Association’s e-forum.  Thank you all for providing your wonderful comments and insight.

Literature search – Thanks to Marlene Vogelsang, Resource Specialist at the Pacific Energy Center who helped identify material for consideration via information database web searches and to Rich Fromberg and Marlene who recently conducted a related search.

Document and protocol reviews – The project team of Rich Fromberg, Manny D’Albora, Robert Davis, John Blessent and Ken Gillespie expended over 500 hours of review time to get this job done.  Thanks team.

Template development – Created by Ken Gillespie with comments from Karl Stum and Dave Sellers.  Many thanks to Mike Kaplan for contributing the initial protocols that were the foundation for this effort.

Database design and construction – Manny D’Albora, Robert Davis, Ken Gillespie and Ernie Limperis.  Ernie is the master database builder of this product. Thanks Ernie.

Project creation and funding – Many thanks to Karl Stum and Dave Sellers who helped write the scope of work.  No job of this size can be completed without funding.  Many thanks to Grant Duhon, Misti Bruceri and Alyssa Newman for their commitment and support.

Database Review - Many thanks to Karl Stum, Dave Sellers, David Shipley, Mary Ann Piette, Norman Bourassa, Keshwar Ramjattan, Jeffrey Rees, Mark Banas, Tav Cummins, Mark Williams, Ron Wilkinson, Tom Anderson, Joan Hitchner, Satu Paiho and Daniel Choiniere for their comments and/or completion of the Upgrade Project Questionnaire.

A.1.3. Protocol Definitions

Our first goal was to create working definitions for categorizing the various types of commissioning test protocols, beginning with those found in ASHRAE Guideline 1-1996.  This process proved to be more difficult than we originally imagined and generated some very interesting dialog.  After four iterations, eight participants, communicating via e-mail and BCA’s e-forum, settled on working definitions for the project. We found that each party had their own reference frame for what commissioning meant to them and how it was to be implemented. 

Process oriented definitions, such as those found in Guideline 1-1996, become less meaningful, even confusing, when applied outside the context of the process defined in the guideline.  In an effort to overcome this, we chose to develop definitions that were more focused on task and less on process.  This provided a fairly straightforward approach to categorization as we developed the review matrix. 

The definitions chosen are as follows:

Verification checks - those full range of physical inspections and checks that are conducted to verify that specific components, equipment, systems, and interfaces between systems conform to a given criteria.  These checks typically verify proper installation, start-up and initial contractor check-out, prior to equipment being functionally tested."

Functional tests - those full range of tests that are conducted to verify that specific components, equipment, systems, and interfaces between systems conform to a given criteria.  These tests are typically used to verify that a sequence of operation is correctly implemented or that a design intent criteria has been met.  They typically are done after equipment is placed in full operation."

Performance tests, which include efficiency, capacity, load, monitoring and M&V or savings protocols, are considered a subset of functional tests.

A.1.4. Literature Search

The team then turned to the task of collecting, cataloging, and indexing publicly available commissioning test protocols.  We also reviewed unique proprietary procedures for reference with respect to the test approach and presentation format, but the actual documents are not included in this product. 

Our scope required that at a minimum the following list of references were to be obtained for review.

·        ASHRAE Guideline 1 – 1996 - The HVAC Commissioning Process

·       Model Commissioning Plan and Guide Specifications, PECI

·       NIST HVAC Functional Inspection and Testing Guide

·       Montgomery County Commissioning Specifications

·       University of Washington Commissioning Specifications

·       United States Army Engineering and Housing Support Center Standard HVAC Control Systems Commissioning and Quality Verification User Guide

·       Multnomah County Standard Commissioning Procedures

·       National Environmental Balancing Bureau Procedural Standards for Building Systems Commissioning

·       ASHRAE Standard 150P – Method of Testing the Performance of Cool Storage Systems

·       ASHRAE Guideline 14P – Measurement of Energy and Demand Savings

·       PG&E’s CES Commissioning Guideline, 1995

·       PG&E’s CoolTools™ Chilled Water Plant Design and Performance Specification Guide

·       PG&E’s Roof-top Unit Performance Analysis Tool Case Study

We used a number of resources to identify and collect other relevant literature, identifying a significant portion of the material for consideration via informational database web searches.  We were greatly surprised to see the vast number of citations that came from the advanced science and research community.  We were also able to utilize the annotated bibliography from a recent CEC PIER project, Method for Determining Measurement Accuracy and Storage Frequency for Improved Building Energy Efficiency, by Rich Fromberg and Steve Blanc.  The team selected and acquired 126 products in electronic and/or hardcopy format.  In addition, we included a few products outside our scope to evaluate how other commissioning environments present their requirements. 

Literature search matrixes and resource lists are provided below.

Major Category Key Words

Building Commissioning

Healthcare Facilities

Building Performance

Hospitals

Building Performance Monitoring

HVAC Systems

Colleges

Institutional Buildings

Commercial Buildings

Laboratories

Commercial Laboratories

Measurement and Verification

Commercial Refrigeration

Office Buildings

Commissioning

Performance Test (ing)

Commissioning Plan

Process Control

Commissioning Specification

Quality Control

Commissioning Report

Re-commissioning

Commissioning Test

Retro-commissioning

Campus Facilities

Testing, Adjusting, and Balancing

Campuses

Universities

Energy Analysis

Verification Test (ing)

Functional Performance Test (ing)

 

 

Subcategory Key Words

Air-handler (Test)

HVAC Air Handling (Test)

Asynchronous Drive (ASD) (Test)

HVAC System Controls (Test)

Boiler (Test)

Hydronic Heating (Test)

Building Automation System (BAS) (Test)

Instrumentation (Test)

Building Pressurization (Test)

Instruments (Test)

Checklist

Lighting Controls (Test)

Chilled Water (Test)

Maintenance (Test)

Chiller (Test)

Measurement (Test)

Clean Room (Test)

Measurement & Verification

Commissioning (Test)

Method of Test

Commissioning Plan

Mode of Operation (Test)

Commissioning Specification

Outdoor Air Ventilation Rate (Test)

Condensing Unit (Test)

Packaged Rooftop Unit (Test)

Control Valve (Test)

Performance Based Codes (Test)

Controls (Test)

Performance Measurement

Cool Storage (Test)

Performance Monitoring

Cooling Tower (Test)

Performance (Test)

Cooling Tower Fan (Test)

Protocol

Damper (Test)

Pump (Test)

Data Collection

Sequence of Operation (Test)

Data Requirements

Test and Balance

Daylighting Controls (Test)

Test Instrumentation

Diagnostic (Test)

Test Method

DDC Controls (Test)

Test Procedure

DX Rooftop Equipment (Test)

Testing

Economizer (Test)

Testing, Adjusting, and Balancing

Efficiency (Test)

Thermal Storage (Test)

Energy Efficiency (Test)

Variable Air Volume  (VAV) Box (Test)

Energy Management & Control System (ECMS) (Test)

Variable Frequency Drive (VFD) (Test)

Energy Management System (EMS) (Test)

Ventilation Effectiveness (Test)

Fan (Test)

Verification (Test)

Functional Performance (Test)

Unitary Fan (Test)

Heat Pump (Test)

 

Test = test, protocol, method of test, test method, test procedure, checklist

Potential Literature Sources: papers, conference proceedings, articles, books, other publications, course materials, personal correspondence

ACEEE (ACEEE.org)

Association of Energy Engineers (AEE)

American Society of Mechanical Engineers

(ASME)

ASHRAE Handbook(s)

ASHRAE Journal

ASHRAE Research projects

ASHRAE Transactions

California Energy Commission

Control Engineering

EPRI HVAC Research Center,

Madison, Wisconsin

EPRI Web (www.epriweb.com)

ESource

Florida Design Initiative (http://fcn.state.fl.us/fdi/index.html)

Heating, Piping and Air Conditioning (HPAC)

International Solar Engineering Conference

IEEE

International Facility Managers Association (IFMA)

Industrial Energy Technology Conference

National Conference on Building Commissioning (PECI)

Northwest Energy Efficiency Alliance

PEC Resource Center

San Diego State University

Texas A&M University-Energy Systems Laboratory

University of Colorado—Joint Center for Energy Management (JCEM)

University of Washington

 

Search Database Sources:

Compendex Engineering Index, Engineering Information, Inc.

Department of Energy Science and Technology

Gale Group Trade & Industry, The Gale Group

ICONDA-International Construction Database, Fraunhofer-IRB

INSPEC, Institution of Electrical Engineers

NTIS, National Technical Information Service

 

A.1.5. Document and Protocol Review Matrix

Once the products were acquired, our next task was to review, categorize, and index them.  To identify, classify and evaluate the protocols as consistently as possible, we developed a detailed review matrix, which evolved as our reviews progressed.  The document and protocol review database includes summaries of the 78 products identified as worthy of further review.  It should be noted that as hard as we tried, the review team did not always make similar judgments in completing the reviews. 

A.1.6. Commissioning Test Protocol Templates

With support from outside experts, the team analyzed the assembled library, identifying the advantages and disadvantages of the various approaches for executing and documenting verification checks and functional testing and to identify gaps where additional protocols are needed.   We developed an improved universal approach through the use protocol templates that would enable standardization and systemization of both testing methods and documentation formats.  While completing the reviews, particular attention was paid to how each protocol was presented.  The review matrix provided a consistent method of identifying the key elements of the commissioning procedure under review.  Did the protocol clearly communicate its requirements and the data to be obtained?  This information was used to guide template development.

Test protocol templates (detailed input forms) with required input fields that can fit into a database were developed based on the working definitions developed previously and the protocol categories identified in the documents and protocol reviews.  In evaluating the various types of protocols, a significant portion of them have been developed by two firms, PECI and Kaplan Engineering, both of which, used various combinations of narrative and tabular procedures to communicate the requirements of a particular test.  It was felt that this approach was suitable depending upon the requirements of the specific template, but that the detailed requirements listed in the Overview should be adhered to.  Instruction sets were developed for each template.  Templates were developed for the following: System Title Page; Verification Checks General Instructions; Operator Interview (retrofit only); Documentation Checks; Equipment Installation Checks; Nameplate Data Checks; Installed Characteristics Checks; Operational Checks; Balance Report Checks; Controls Related Checks; General Run Checks; Verification Completion Log; Functional Tests General Instructions; Comfort/Air Flow Tests; Forced Response Tests; Operational Trend Tests; Sequence of Operation Tests; Performance Tests.  It is anticipated that any number of these templates could be grouped together in systems procedure.  

Analyzing the database for potential gaps revealed some interesting results.  Remembering that only a portion of the current literature has been reviewed and that a more detailed analysis could reveal different results, we believe that public domain protocols for the following sub-systems and protocol types need to be developed: condenser water treatment; economizer control; outside air ventilation, building pressurization/infiltration; HVAC air handler capacity; terminal unit dumping; energy management system hardware & software. 

A.1.7. CTPL Information Database Tool

The ultimate project goal was to assemble a Customer Support Tool based on the resources developed by the preceding tasks.  Tool specifications were as follows:

·       Search for information based on key words and other criteria such as test protocol technique or approach and/or equipment type, author, source, etc.

·       The original data in the infobase library shall be totally protected so that the user cannot modify it in any way other than by creating and editing copies.

·       Paste test protocols selected from the Library of Non-Copyrighted Documents into template forms for editing and/or modification and use by the user of the tool. 

·       Save modified procedures to the New Protocol Library; index for later use.

·       Print selected procedures directly from the infobase without conversion to another file format for processing.

·       Provide instructions for using the infobase.

·       Provide a brief discussion of the project and the intended use of this product.

·       Allow the infobase data structure to be expanded to include additional categories for new procedures developed by the user and/or to support additional procedures.

The database plan provides for four primary components: a document and protocol review database, a library of existing non-copyrighted commissioning related documents, a protocol template database and a New Protocol Library.  The review database, the library of protocols, the user protocol library and a revised protocol templates document have been completed and are included in the tool.