search

mechyai / RL-BCA

Repo for RL algorithm design and testing of BCA (building control agent) HVAC on EnergyPlus building energy simulator using Python API/EMS
Apache License 2.0
11 stars 4 forks source link

Advice on connection to OpenAI Gym Stable Baseline #6

Open Farz5 opened 2 years ago

Farz5 commented 2 years ago

Hey mechyAi,

I have been using your code and wanted to thank you for the great work! I am trying to connect stable baselines to Emspy. I was wondering if you have any tips or similar codes to share? I would really appreciate your opinion and help!

Best, Farzan

mechyai commented 2 years ago

Hello Farzan,

Thanks for reaching out, I'm glad its been working for you!

This repo is very much under development, and not in a release-ready state, still I wanted to make it accessible for willing people like yourself. I have had a nightmare keeping the repo updated, I haven't pushed to it since September '21, despite my progress. I would like to get a stable version out, but I am too busy with my Master's research to put the TLC into this repo that it deserves.

I will see what I can do in the coming week to push the newest version and provide some example usage.

You will have to send me what you're trying to do.

The agent/environment interaction-style I chose for EmsPy is very different from the step() API style of Open AI's Gym API. I did this because (1) EnergyPlus requires callback function to the running simulation which I think doesn't adapt well to the step() API flow, and (2) it allows maximal flexibility in: how you set up your E+ building models, when you collect data from calling points, how you organize your RL agent's actions and observations, etc. This flexibility naturally leads to more complexity with using the API . Also, I just couldn't figure out a natural way to adapt things to the Gym API format, but I am sure there is a way and I am sure you could try figure it out.

My guess is that to turn EmsPy into a Gym API, you would need to make strict decisions on how you set up your building model, calling points, state space, action space, and how you implement your actuation & observation functions - essentially an API wrapping EmsPy to remove all the flexibility. Just like how the Gym API's are very rigid and only expose certain flexibility and MDP variables.

Hope this helps,

Chris

On Fri, Mar 11, 2022 at 4:11 PM Farz5 @.***> wrote:

Hey mechyAi,

I have been using your code and wanted to thank you for the great work! I am trying to connect stable baselines to Emspy. I was wondering if you have any tips or similar codes to share? I would really appreciate your opinion and help!

Best, Farzan

— Reply to this email directly, view it on GitHub https://github.com/mechyai/RL-BCA/issues/6, or unsubscribe https://github.com/notifications/unsubscribe-auth/APTF5CWXGPXFNGMTJANLXW3U7OZHZANCNFSM5QQWI7LA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

mechyai commented 2 years ago

@Farz5 See https://github.com/mechyai/RL-EmsPy for updated code, repo, and some examples