Established in 1998, Customized Energy Solutions manages electrical generation assets such as wind farms, solar farms, hydroelectric plants, and many other types of IPPs (Independent Power Producers) for their industrial clients. CES operates a central control room, in Philadelphia, handling assets distributed throughout the United States and Canada. Reliable remote communication is key to a successful operation and operators must be aware of communication or equipment trouble as soon as possible so they can orchestrate a smooth and orderly recovery. Kevin J. Komara, P.E, director of data acquisition services, is responsible for operations of real-time data acquisition systems at CES and the development of new control system technologies and strategies.

“We are looking forward to deploying OPC Expert on new installations as they come online,” Komara said.

In a specific installation, CES uses their Philadelphia operations center to manage a client with 8 remote wind farms spread over multiple ISOs (Independent System Operators). CES operators use their SCADA HMI to connect to eight remote wind farms.  Each wind farm has two OPC servers: a GE WFMS (Wind Farm Management System) OPC server and a Schweitzer Engineering  Labs SEL substation controllers OPC server. In total, they receive information from 16 individual OPC servers.

This operation has several challenges:

  • Initial setup times introduced unacceptable project risk. Distributed operations, spread over many remote sites, included a multitude of different Windows Workgroups and Domains. These different computer management and security systems increased project integration time and difficulty.
  • Communication issues were forcing CES to reconsider their architecture as they caused endless troubles for their SCADA HMI, which depends on a reliable communication infrastructure
    • Network outages due to brief TCP/IP timeouts (or total communication loss), hardware failures, or system hardware upgrades, were occurring at random times, preventing the HMI from recovering gracefully (i.e. without manual intervention).
    • Unconventional OPC server behavior caused the HMI connection problems and ambiguous data interpretation.  Although these servers were fully compliant, their approach to data provision methods were rather unique. For example, some data items reported value updates when these were not required.  Some items were grouped together in a single compound item, which then had to be broken back down to multiple items. This was common with on/off items for switches.
    • Unpredictable OPC server failures caused some items to stop updating as expected and these needed to be restarted.
  • System recovery after failures forced CES operations to use costly and inconvenient manual intervention at all hours including late evenings and early mornings. This was unacceptable for reliable 24/7 operations.

Figure 1: CES used OPC Expert to aggregate data from all OPC servers, provide calculations, and inform of communication problems thereby improving communication reliability and fault recovery

Kevin was aware of OPC Expert since he regularly used it as an OPC communication troubleshooting tool in previous installations 8 years prior.  He used OPC Expert to aggregate data from all OPC servers, provide calculations, and inform of communication problems thereby improving communication reliability and fault recovery.  As Kevin put it “OPC Expert works as a traffic cop for the OPC servers.”  Using OPC Expert, he changed his system architecture:

  • Single OPC server connection: Instead of having the HMI manage all remote OPC connections across the WAN, CES configured OPC Expert to handle all OPC server connections. This enabled the HMI to avoid having to manage multiple OPC servers and their communication faults.  Instead, OPC Expert provides a single data platform.  When communication is troublesome, OPC Expert marks faulty information as bad quality, thus isolating the HMI from low-level communication management. This immediately improved HMI operation.
  • No tunnels: due to OPC Expert’s ability to quickly recover from communication faults, Kevin avoided the use of tunnels, which would require many extra licenses and adding even more moving parts. OPC Expert handles all communication over the WAN and negotiates secure communication between Windows Workgroups and Domains and across firewalls.
  • Calculations: using OPC Expert’s calculations, CES created new items from existing data. For example, some calculations were used to create “bit tags”, which took information from one item and created multiple Boolean (on/off) items. Other calculations converted existing item data to engineering units.
  • System monitoring: OPC Expert provided new system monitoring items for each OPC server (such as last callback times), enabling CES to receive immediate failure alerts.

“After deploying OPC Expert’s provisions, our HMI is now able to reliably recover from underlying system communication problems and provide timely and accurate information to operators,” said Kevin. “We received very helpful technical support and training from OPC Expert and have been running the system reliably for months.”

Based on their project success, Kevin purchased additional licenses for use in other projects. “We are looking forward to deploying OPC Expert on new installations as they come online,” he added.