Originally Published in Project Haystack Connections Magazine, January 2022 Issue

Adapt or Die: A phrase that has been used to describe many situations including in business. Technological advancements in the past decade alone have changed our lives and businesses in a myriad of ways. The design and construction industry has seen this firsthand, most notably with building information modeling, but advancements are happening every day. At Epsten Group, only 10 years ago we were using spreadsheets for tracking commissioning issues with engineers and contractors during design and construction. This seems arcane now that well-established, cloud-based software exists that help with this function and at a fraction of the cost of labor spent managing spreadsheets and email exchange.

One thing that hasn’t changed much in the commissioning of building systems is functional testing. We test mechanical, electrical and plumbing systems to ensure that they function as intended. One could argue that this is a commissioning provider’s highest priority. And we have done this manually (on the job site in front of the equipment) for decades. This was necessary, and continues to be so, for many systems we are hired to test. However, technological advances combined with standardized naming/identification of data points within building automation systems has opened a door to an automated approach to testing of repetitive, terminal-level systems.

A typical commissioning scope includes HVAC systems and associated controls. Regularly we are tasked with sampling “like” equipment such as terminal units, fan coil units, etc., which are high in quantity on a given project. Our firm works regularly on new buildings with more than 100 terminal units which have application-level controllers on them to perform relatively simple and repeatable functions, modulate for space temperature control, adjust for CO2 in the space, etc. While the sequences and controls for these devices can be quite simple, they both have a direct and indirect impact on occupant comfort, building energy use, and indoor air quality, so they cannot be ignored. However, a sampling rate during functional testing of 100% is often unjustifiable given the quantity of equipment and limited complexity. Therefore, we often sample at some rate between 10% and 30% for terminal devices, which keeps costs down and is statistically viable.

As commissioning agents, we also spend a lot of our testing time on troubleshooting operational issues which extends to these terminal devices. Malfunction is not saved just for the more complex systems such as centralized air handling units, chiller plants and integrated lighting control systems. But, we are rarely able to account for this time when putting together proposals. Otherwise, our clients might think we are overcharging or believe they can get a better deal from another firm. This presents further opportunity for leveraging the technology at our fingertips to test repetitive, less sophisticated systems automatically and to do so at scale. Haystack tagging, for example, really helps to apply similar analytics across multiple projects by coding the analytics to look for specific tags. From a software development viewpoint, this is very powerful and lowers the cost of deploying automated testing and analytics. As a commissioning provider, we don’t have to really understand how tagging works; we simply see lower costs from our technology partner because of the benefits of Haystack data normalization.

Whereas the commissioning agent is typically sitting with a controls technician at some front-end connection and adjusting setpoints to manually check performance, automated functional testing removes this step. If we typically sample terminal units with manual testing, why can’t we test every terminal unit with automated functional testing? Now we can. And we have already done so on numerous projects at the end of construction, before owner occupancy.

So how does it work? For new buildings, we connect a cellular-enabled encrypted gateway to the BAS at the network- or field-controller level. That gateway then connects to the cloud where our automated tests live. In our case, we utilize an Automated FPT tool called OTTO, which houses our automated FPTs in a time-based program or sequence. Over the course of several hours, often run at night, our FPTs are executed on terminal devices all at once (or sometimes in smaller batches, but always numerous devices in a given window of time). At the conclusion of the testing, we automatically receive a summary report of the results, including pass and fail steps for each terminal device. And then the analysis begins of the fail steps.

This narrows the focus to the problematic terminal devices out of the 100% sample and expedites the generation of commissioning issues and subsequent resolution by the responsible contractor. Where manual testing might take days, plus time to do troubleshooting, it is effectively cut to a single overnight activity with a significantly improved sample size. For most commissioning agents, the schedule is not our friend. We are shoe-horned in at the end of the project and gaining back even a couple of days is of tremendous value, not only to us, but also our clients and their general contractors.

Additionally, testing is typically being conducted prior to the client’s IT network being in place. Therefore, we have a great opportunity to have early access to the BAS and associated data that we normally have to request from the controls contractor. That data is often provided in simple .csv file outputs that require a lot of massaging to analyze, not to mention it is only a snapshot in time. This arrangement also preempts the often-onerous security requirements needed to access the client’s BAS network remotely. However, having the IT network up and operational is not a bad thing, simply another piece of the puzzle. On the case studies described below, we worked through the client’s IT networks, which did have security requirements, to satisfy but was done so successfully.

A couple of examples include the recently opened ASHRAE Headquarters in Peachtree Corners, Georgia and the new Academic Learning Center at Kennesaw State University. The ASHRAE HQ project was one of the first where our team attempted automated functional testing and was conducted as a supplement to manual functional testing to verify the potential of the automated FPT approach. The building provides thermal comfort to occupants using radiant heating and cooling ceiling panels across 75 unique zones which are served by an air-cooled heat pump chiller. These radiant zones are repetitive in nature and therefore are ideal candidates for automated testing. Our test scripts were automated in the cloud and overrode the applicable BAS setpoints during the early morning hours to test cooling, heating, and satisfied conditions as well as interaction with the BAS-controlled ceiling fans in each zone. We were able to verify anomalies and failures of individual zones and then focus our attention on those problem zones with the controls contractor immediately.

The KSU Academic Learning Center is a more standard HVAC infrastructure with a combination of 122 terminal units and fan-powered terminal units with hydronic reheat all served by central-station air handling units. Testing began prior to the owner’s IT network installation and therefore we utilized a cellular gateway for testing on each floor, but the process of testing was the same. The IT network was brought online near the end of our functional testing, and we transitioned to utilizing that network, so this project used a hybrid approach. In the end, all that was required on our part was the analysis of collated test results data and the correlated trend data.

In conclusion, technology advances plus standardization initiatives like Project Haystack have enabled commissioning providers to be more efficient with their time while maintaining a high level of service plus additional benefits to their clients. We have embraced automated functional testing as a result and will look to expand its use over time.