Digitalized-Energy-Systems / opfgym

A gymnasium-compatible framework to create reinforcement learning (RL) environment for solving the optimal power flow (OPF) problem. Contains five OPF benchmark environments for comparable research.
https://opf-gym.readthedocs.io
MIT License
2 stars 0 forks source link

Differentiate between observations and states #29

Closed thomaswolgast closed 11 hours ago

thomaswolgast commented 4 days ago

Currently, only the observation space gets defined. The same space is also used for sampling, which can result in side effects. For example, if the observation space is reduced (partial obs), the non-observed state channels might not sample correctly anymore.

Approach: Explicitly define state space for sampling and observation space for observations. Further things to to:

Related to #10 and #8.

thomaswolgast commented 11 hours ago

Do not store state in dict to save computation time. Maybe add this later, but then with popgym support (https://popgym.readthedocs.io/en/latest/autoapi/popgym/core/env/index.html).

Do not include results in state. They are not required to predict the next state, so they are not part of the Markov state.