In tropical countries, AC accounts for almost half of the total residential building energy consumption. In university campuses, it’s even more significant because of the dynamics of the buildings. Using the AC isn’t just about costs — it’s about comfort, learning environment, and environmental impact. In university campuses the AC can be a wasteful resource if it is not used to adequately cool a classroom. Meaning that the classroom may be too cool for school. Too cool for the current number of occupants.
NCTU established the Smart Campus project where industrial and academic research worked together to improve the campus environment. There were may projects implementing efficient wireless networks to implementing efficient building controls. The CC Wang Lab took up the challenge of making a classroom more energy efficient.
Improving the classroom environment
The problem: The AC temperature is set by the professor, TA, class leader, etc but it doesn’t consider occupancy, weather, or activity levels. It’s just whoever’s in charge picking what ‘feels right’ to them, and only gets changed when enough people complain about being too hot or too cold.
Why is it important to improve the classroom environment? Having a suitable thermal comfort and good air quality promotes a good learning environment. If the classroom is too hot or too cold, it will cause lack of concentration. If the concentration level were to get too high it can cause drowsiness, lack of concentration, and poor cognitive control. So it is important to keep the concentration below a certain threshold.
The plan: Use smart systems to control the set-point of the AC while taking into consideration the number of occupants while also being energy efficient.
Implementation
We’re off to the races! Before you implement a something, you should first look for existing research/solutions and see their approach. Perhaps it has already been solved before? After doing literature review we found that there was research similar to our plan but not in the same manner we wanted to (which was good news for us, research gap!).
At the time there was a major news from DeepMind where they were able to train a neural network to play a variety of games with reinforcement learning (RL). With RL you basically have an environment and you have an “agent” which you provide the agent with actions it can take. The agent can then make any of those actions and observe how those actions impact the environment. Furthermore, you create a “reward function” that given the new state the agent will receive a reward. The agent will try to take the best actions to receive the highest rewards through each epoch.
We clearly saw how the environment would be the classroom, the actions would be the AC set-point and exhaust fan controls. And that the reward function would be to optimize the indoor thermal environment whilst keeping the energy consumption low.
Reward function
To create a meaningful reward function, we had to understand what factors are relevant to the indoor thermal environment. One important term I learned is thermal comfort, the ASHRAE 55 defines thermal-comfort as “that condition of mind that expresses satisfaction with the thermal environment and is assessed by subject evaluation”. Another important term I learned is the Predicted Mean Value (PMV) which is a model to estimate if the conditions of a thermal environment are acceptable to a majority of the occupants within that space. Another important factor was levels.
With the previously mentioned terms [1] came up with a base reward function (refer to their reading for full details):
The environment
The classroom could up to 72 occupants, it is located in NCTU’s Mechanical Engineering building. The classroom had a LoRa network with sensors and remote controls for the AC and exhaust fans:
We needed to build an environment which the agent could interact with for offline training. We read Wei et al [2] and used their same approach of simulation and training: EnergyPlus and BCVTB for simulation and double Q-Networks.
After the offline training was done, we deployed the neural network onto the classroom environment and we found that there was a reduction of energy consumption, reduction of while the thermal comfort of the classroom was kept in an acceptable point.
Results
It was great to see the classroom environment being controlled by the agent. I would smile (almost) every time that it would change the environment in the middle of a class. Here are some results on how we optimized the classroom:
- Developed a simulation for offline training of the classroom environment [1]
- Balanced energy consumption, thermal environment and indoor air quality [1]
- Controlled the environment during different activities (lecturing and examination) [2]
- Reduced by 24% with the agent [2]
- Reduced energy consumption by 19% and 15% (when compared to fixed temperatures) [2]
- Implemented LED control based upon the occupants in the classroom [4]
References
[1] Valladares, W., Galindo, M., Gutiérrez, J., Wu, W. C., Liao, K. K., Liao, J. C., Lu, K. C., & Wang, C. C. (2019). Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithm. Building and Environment, 155(December 2018), 105–117. https://doi.org/10.1016/j.buildenv.2019.03.038
[2] Wei, T., Wang, Y., & Zhu, Q. (2017). Deep Reinforcement Learning for Building HVAC Control. Proceedings of the 54th Annual Design Automation Conference 2017 on - DAC ’17, 2, 1–6. https://doi.org/10.1145/3061639.3062224
[3] Kuan-Heng Yu, Yi-An Chen, Emanuel Jaimes, Wu-Chieh Wu, Kuo-Kai Liao, Jen-Chung Liao, Kuang-Chin Lu, Wen-Jenn Sheu, Chi-Chuan Wang (2021). Optimization of Thermal Comfort, Indoor Quality, and Energy-Saving in Campus Classroom through Deep Q Learning. Case Studies in Thermal Engineering, https://linkinghub.elsevier.com/retrieve/pii/S2214157X21000058
[4] Kuan-Heng Yu, Emanuel Jaimes, Chi-Chuan Wang. (2020) AI Based Energy Optimization in Association With Class Environment. https://doi.org/10.1115/ES2020-1696