Thermocloud uses a two-tiered learning approach to dissect the huge problem space so that a very large number of Thermocloud clients can be supported in a reliable and responsive manner. The first tier leaning involves identifying the right server capable of performing the best optimization for a specific client operation under a specific operating condition. The second tier learning happens at the chosen server, wherein the server determines the best strategy to help learn and obtain optimal client operation.
Cloud Architecture

Thermocloud clients and servers communicate with each other using HTTPS. To communicate with the server, a client sends a request consisting of the Active ECU, Passive ECU, the temperature, humidity, and schedule conditions. The Active ECU and Passive ECU are derived from the mathematical model specific to the client’s system and zone characteristics. This prevents the system from obtaining any additional information from the user, to maintain their privacy, and store less unnecessary information on the server.

The request is sent to a designated server at its provisioned address. The server is identified using tier-1 clustering algorithm based on the client’s request and operating conditions. The client uses this routing tier-1 algorithm to communicate with the optimal tier-1 server, which uses a collaborative strategy, probe, transit, or sustain, to determine an ideal estimate based on the current schedule and operating conditions.

The server uses a hybrid algorithm to determine the optimal strategy (grad-learn or neural) based on the current number of entries. The client uses the local algorithms to optimize the operation built using server’s recommendations and adapt it to its unique thermal profile if necessary. After obtaining the optimal operation, a report of the optimal stage, time, and duration is then sent back by the client. Server uses this report to train or refine the grad-learn and neural algorithm for these conditions. This is shown as follows:

Cloud Clustering

Thermocloud architecture allows a very large number of thermostat clients to collaborate their operations. As more clients participate, more clients are able to take advantage of the information learned by other clients and that enables improving the performance of all clients. However, a huge number of clients cannot be handled by a single server. Therefore, Thermocloud cloud architecture uses a cluster of server nodes that optimize client operations by collaborating client operations for similar clients and operating under similar conditions. The clustering of operations is performed automatically and is learned in an unsupervised fashion.

Client Similarity

Each client operation uses an Active ECU (AECU) and a Passive ECU (PECU). The AECU models the ability of the HVAC system at the client to heat or cool the zone by a unit degree and abstracts the numerous factors that affect this change in temperature. The PECU models the rate at which the client’s zone changes temperature in absence of the HVAC system operations. These factors effectively represent the thermal characteristics of the client’s zone and the ability of the HVAC system to operate in the zone. Clients with similar AECU and PECU can therefore be regarded as similar in regards to their effectiveness of performing the HVAC operations and therefore, can effectively collaborate for these operations.

Routing in Cluster

Clients with similar AECU-PECU are represented with similar shape and colored icon in the adjoining figure. These clients consult a routing provider node in the cloud to obtain a current routing map. The routing map enables the client to identify the cloud server nodes that are capable of helping the client collaborate with its peers for a specific operation under the current operating conditions. Routing is performed based on many parameters for each type of operation, such as the AECU, PECU, operation mode – heating/cooling, outside temperature, outside humidity, inside temperature, inside humidity, comfort level, interval ranges.

The routing map contains a routing entry for each routing parameter for each server node in the cloud. The routing entry for the parameter comprises a low and high value threshold that is currently handled by the server node for the operation. All server nodes in the cluster collaborate with each other so that the routing map remains consistent across the cloud. Moreover, the clients are provisioned with routing provider nodes that can be queried by the client at any time to obtain the current routing map. The routing map is a dynamic construct and changes as the cloud experiences clients and operations in new operating conditions. Such changes are lazily made available to the clients on an as needed basis.

Cluster Expansion

Thermocloud clustering is automatic and performed using concepts of unsupervised learning. Initially a set of cluster nodes is designated to perform processing for a specific range of AECU-PECU, thereby ensuring that clients with similar operational characteristics are clustered together. When a large number of nodes are available in the cluster, many nodes may operate on clients with similar AECU-PECU ranges. One technique that may be used for clustering is to lazily build and expand cluster based on local proximity of critical operating condition parameter values.

When a client encounters operating conditions for an operation that cannot be routed to any server node, it triggers cluster learning. Using an unsupervised learning algorithm, a server node capable for handling this new operating condition may be determined and the server expands its operating range for the new operating condition. The new sever can update its routing map and similar requests can be propagated henceforth to this new server.

A variation of k-means unsupervised learning algorithm may be used to obtain an initial cluster and refine such cluster during redistributions. Entire AECU-PECU range can be partitioned using domain-specific knowledge of prevalent AECU-PECU ranges and assigned to cluster nodes. A weighted Euclidean distance can be computed for the new operating condition O. Centroid for each node can be computed using the information available in the routing map and the closest cluster node A can be identified for the new operating condition.

In order to facilitate a quick routing-map based client routing, a strict parameter range spanning from a low to high threshold for each routing parameter must be maintained for each cluster node. If the new condition O is closest to A and if A can expand its range of affected parameters without overlapping any other node X, node A’s ranges are expanded to include new condition O. In case there is an overlap with an existing node, the existing node can assume ownership of the new condition O.

Consider an operation Q with \( n \) parameters defined as \( Q = [q_1, q_2, \cdots q_n] \), where \(q_i, 1 \le i \le n\) represents value of an operating condition parameter \( i \). The operation request is routed in a cloud with servers, each server characterised by a centroid \( C_{sj} = [c_1, \cdots c_n] \), where \( c_i \) represents value of the operation condition parameter \( i \) at server \( S_j \)'s centroid, based on the routing map entry for server \( S_j \). Each parameter \( i \) is assigned a unique weight \( W_i \). The client routes the request for operation Q to the closest server \( S_j \), wherein the distance \( D_j \) of the operation Q to the server \( S_j \) is given by:

\[ D_j = \frac{1}{N} \sqrt{\sum_{1}^{n} W_i(q_i - c_i)}\]

It must be noted that the centroid \( C_{sj} \) gets incrementally altered as the routing entry of server \( S_j \) expands on learning new ranges for the operating condition parameters as it encounters new definitions of operations.

This clustering design lends itself to many other advanced operations such as cluster redistribution, efficient neural model retraining on redistribution, splitting cluster nodes and offer a pragmatic management of a very highly scalable cluster of server nodes supporting huge number of clients.

Collaborative Predictions

This is the second tier of the two-tiered learning approach, wherein the server uses a strategy-based approach to predict optimal operations based on knowledge acquired from other collaborating clients. The strategies highlighted here are merely a few of the many possible strategies possible and this aspect of the project is expected to evolve very heavily as described by the patented technology. This design approach allows future research work and advancements in collaboration and prediction techniques to continue making these collaborative predictions even more effective.

Learning Strategies

Thermocloud cloud-based collaborative prediction design allows using many learning strategies to improve the efficacy of thermostat operations. The following learning strategies are the initial focus of this project and many more strategies may follow in the future:

  1. GradLearn: This strategy takes the appropriate data point from the cloud and uses that data point as if it was a value learned by the local client system. Thermocloud learns progressively using domain-specific algorithms to identify most relevant entry as learned from collaborating clients for different operating conditions.
  2. Neural: This strategy uses a deep learning neural network to learn optimal parameters for a specific type of operation based on (re)training based on optimal parameters reported in the feedback obtained in the past from collaborating clients for the operation.
  3. Hybrid: This strategy attempts to balance best outcomes from other strategies. It uses criteria such as amount of training and past performance reliability under similar conditions to decide optimal strategy. For example, it may prefer neural strategy for conditions that saw extensive past training but may opt for grad-learn when past performance indicates neural was not always reliable for these conditions.
GradLearn Strategy

This strategy takes the appropriate data point from the cloud and uses that data point as if it was a value learned by the system. The Active ECU and Passive ECU are unique for each category of houses (insulation, stories, size, etc). Therefore, obtaining these ECUs and creating an approximate range of values enables inferring the category specifics, as demonstrated below. GradLearn strategy uses domain-specific algorithms to determine optimal duration, time, and stage corresponding to the category, and uses these values in the client thermostat. The client thermostat tests these values and sees its effectiveness. It optimizes the value for that thermostat and sends that thermostat’s optimal duration, time, and stage. It then factors into the average, as necessary, to further enhance the values.

This algorithm projects relevant entries of past optimal performance for the requested operating conditions and then uses a variant of K-nearest-neighbor algorithm with Weighted Euclidean distance to identify optimal entry. It identifies the K closest entries to the current operation, and finds the one with the least Weighted Euclidean distance. This essentially minimizes the distance \( \Delta D \) between an existing entry \( E = [e_1, e_2, \cdots e_n] \) and the current operation \( Q = [q_1, q_2, \cdots q_n] \), where \(e_i\) and \(q_i\) represents value of operating condition parameter \( i, 1 \le i \le n \) in the entry and operation resp.

\[ \Delta D = \sqrt {\left( \sum_{i=0}^{n} w_i(q_i - e_i)^2 \right)}\]

Entry with minimum \( \Delta D \) is the closest entry to the operation and hence is most relevant. However, if no past performance optimal entries are available, the client must use the local algorithms to learn the optimal operation.

GradLearn Strategy Algorithm
  1. Using the target temperature, active/passive ECU, operating conditions project past performance entries that would be relevant for computing the optimal operation for the requested conditions.
  2. If there are no entries, instruct the client to use local algorithm to begin learning optimal operation for these operating conditions.
  3. Use a K-nearest-neighbor algorithm with weighted Euclidean distance to find the optimal past entry that matches the requested operation. Compute a recommendation operation based on the optimal past entry and return to client.
  4. When client reports an optimal entry for an operating conditions determine how learning must be performed for these conditions:
    1. If sufficient past performance entries are absent, mark for cost optimization by widening within schedule range. Subsequent reports must mark for bumping comfort. On subsequent client requests, instruct client as marked here.
    2. If learning completed and report indicates suboptimal performance, mark for retraining.
    3. Commit and learn best parameters used for this optimal entry under the current operating conditions.
Neural Strategy

This strategy uses a deep-learning neural network for each local operation whose prediction is required. Currently, this includes a network to predict optimal values for transit and sustain operations under varying operating conditions.

It consists an input layer, with different inputs for transit or sustain, multiple hidden layers with various weights and biases, and an output layer with the optimal stage, time, and duration.

A trained neural network essentially regresses operation parameters based on past training from other collaborating clients that operated in similar operating conditions. With sufficient training, this strategy is expected to excel as it is based on actual or close to actual operation parameters from clients who experienced actual optimal results.

Neural Strategy Algorithm
  1. Obtain the target temperature, active/passive ECU, operating conditions, etc from the local thermostat.
  2. Train neural network by creating the network nodes, a square mean error loss function, and use gradient descent for various epochs of training data.
  3. Use the values obtained from the cloud and run the required strategy according to those values. Use the appropriate stage and scale the duration to the operation. The network input and output parameters must be conditioned appropriately to be able to work in accordance with requirements of the neural network model implementation
  4. If above step fails to compute required operation, delegate processing to alternative strategies.
  5. When client reports optimal operation, accumulate this for model retraining and schedule a retraining when sufficient entries are available.
Hybrid Strategy

This strategy combines the Neural and GradLearn strategies by using the Neural strategy when there was adequate training for those conditions and reverting to GradLearn strategy in absence of adequate training.

It bootstraps with GradLearn strategy and any feedback from collaborating clients also provides continuous training for the neural strategy. Once there is an sufficient training, the neural strategy can start predicting client requests.

Upon deterioration in performance, neural strategy is deemed unreliable for those conditions and GradLearn strategy is used until sufficient (re)training makes neural predictions reliable once again.

Hybrid Strategy Algorithm
  1. Obtain the target temperature, active/passive ECU, operating conditions, etc from the local thermostat.
  2. Strategy Evaluator: Evaluate the optimal strategy by identifying which strategy uses least number of ECUs under the current operating conditions. If exact strategy under exact operating conditions are not available, use regression to identify these conditions.
  3. Strategy Invoker: Use the strategy to heat/cool house according to output. Adjust operation locally, as done in neural/GradLearn strategy.
  4. Composer: Every x entries, identify which strategy is most optimal under most operating conditions. Compose these two strategies to find the optimal strategy, using shown by the graph below.

The following graph represents the expected curve between Grad-Learn and Neural, where red represents effectiveness of grad-learn and blue represents effectiveness of neural. There is a certain point(approximetly 500 entries) where grad-learn exceeds neural. The goal is to use the most optimal strategy with the current amount of training data.