search

corymosiman12 / dragg

Distributed Resource AGGregation (DRAGG) implements centralized MPC for residential buildings using an aggregator and residential building owner (RBO) model
MIT License
5 stars 5 forks source link

beta update for longer simulation times and RL pricing #5

Closed apigott closed 4 years ago

apigott commented 4 years ago

This update enables several changes to the DRAGG core functionality and allows for integration of the DRAGG package for Home Energy Management Systems (HEMS) with a generalized reward price signal design (in this case OpenAI Gym for Reinforcement Learning).

Changes to the HEMS in DRAGG include:

  1. Introduction of sub hourly minimum runtimes in [homes.hems.sub_subhourly_steps] where the homes can optimize binary on/off decisions for their subsystems on discrete timescales independent of the aggregator agent. For example, the aggregator agent can analyze the grid performance on the hourly scale while binary on/off of the home HVAC can be determined on a 15 minute scale. Effectively giving the homes a discrete set of duty cycles to choose from.

  2. Error handling resolves errors in the MPC solver by implementing the control decisions of the previous timesteps. Since the homes have perfect knowledge of the environment over the future (weather, GHI, waterdraws) the environmental variables such as Tin and Twh from those previous timesteps are considered valid.

  3. Homes now include randomized waterdraws which are centered around random times of the day and varied stochastically. For example, home Crystal-RXXFA might see a water draw between 6pm and 8pm every day of approximately the same size. (For now homes have perfect knowledge of water draws within the MPC horizon.

  4. MPCCalc class now initializes a single house instance with a dedicated CVXPY object at the beginning of the Aggregator and community initialization rather than resetting the CVXPY object for each house in the community. This allows for parallelization of the homes via Pathos.

  5. MPCCalc class now logs homes individually (required by Pathos parallelization).

Changes to the Aggregator in DRAGG include:

  1. Added a simplified (purely linear) P_grid response "rl_simplified" to benchmark performance of the RL price design and test integration to OpenAI Gym with the same syntax but without HEMS dynamics.

  2. Removed non-MPC specific base case scenario. Instead run with [homes.hems.prediction_horizon] = 0 or 1.

  3. Added capability for non-MPC and MPC enabled homes in the same community using [community.total_number_homes] = [MPC/responsive homes, non-MPC/non-responsive homes], etc.

  4. Write data every timestep (due to OpenAI integration issue). Otherwise write data at intermediate checkpoint intervals (ex. every 100 timesteps) so that runs can be quit before termination without losing results.

General changes to the DRAGG package include:

  1. Parallelization of the HEMS MPC optimizations via Pathos package.

  2. Change the syntax of the config file from JSON to TOML to allow for comments in the config file. (Now config.toml)

  3. Redis-server in a singleton class such that only one redis connection is open locally.

  4. Change logger to be a single initialization such that only one logger is open per class instance (e.g. one per home, one per Aggregator, etc)

corymosiman12 commented 4 years ago

@apigott - this is now merged. also gave you access, so feel free to just use this repo. Let's try to keep the master branch in a working state. I'm gonna try to start from scratch using the README and see if I can get it to work. Maybe we can add some docs to help explain better as well. kudos on all the work!