Monte-Carlo-based simulation

[Webbah]

To represent real world more accurately, define parameters as distribution (gaussian, normal).

• In every simulation step, drag from distribution and simulate, calculate performance.
• Do this e.g. 10 times with the same controller parameter set
• Use mean value of the performance in the optimizer associated with the controller parameter set

Parameters:

• R, L, C
• vDC?
• Noise
• Alter all phases at once
• Alter L1 L2 and L3 separately

(Implementation will be done in #59 )

[wallscheid]

Variant: Compare mean or min (worst-case) over the sampled performances to robustify the SafeOpt pipeline.

[Webbah]

_nMC: number of Monte-Carlo samples as input to runner

runner

• [x] Additional n_MC-loop

Env: Every n_MC simulation: new parameters are drawn

• [x] update env parameters in env.reset()

• [x] get parameter value from distribution (using partial?)

  • additional class Load in main skript

Agent: only has to update after n_MC simulations Every n_MC simulation: new performance is calculated After n_MC -> average performance used for optimization to find next controller parameters

• [x] counter to optimize only after n_MC
  • not needed:
  • in MC-loop:
    • observe function is called with terminate = False to track return
    • agent.performance is called to store n_MC performances is array
  • After MC-loop
    • mean of performance array is calculated and taken to:
    • observe is called with terminate = True to update controller params

[Webbah]

Changed env.reset() to initialize model parameters with different values using model_param : https://github.com/upb-lea/openmodelica-microgrid-gym/blob/c845d1a7769c67ba203bbfeeaa510a09e4cde95d/openmodelica_microgrid_gym/env/modelica.py#L263

Even solves the problem that the first parameter value (before the first step) is correct and avoids loadsteps in the beginning.

@stheid better possibility or other suggestion? Or where else are the initial parameters (from python) set yet?

---

Problem with additional loadstep?

• Parameter drawn from random distribution
• if t > t_loadstep: gain*parameter, but parameter unknown...

[stheid]

I mean, OpenModelicaParameters can be functions of any type, so this is not an issue from an implementation state. But you also mean to addidionally parametrize the rest of the environment, right?

[Webbah]

First Additionally: Abstract agent class EpisodicLearnerAgent(Agent) -> safeotp inherits from EpisodicLearnerAgent and staticctrlAgent

• has properties:
  • performance (@ ...)
• has methode:
  • update_params()

Execution:

• new runner class 3 Loops: Episode, n_MC, step Every n_MC:
• store agent.episode_reward (NAME CHANGE: Return!) & agent.interations -> cal episodic_Performance -> store in np.array
• reset agent & env
  • NO! Controller are resetted in prepare_episode! aget.reset() would reset the GP!

agent.observe always call with done = false -> update_params NEVER called! After n_MC runs of n_MC loop agent.performance = mean(episodic_Performance) Call agent.update_params explicitly

-> so present agent and env don't have to be modified

[Webbah]

Problem with additional Load class in main script:

• R,L parameter get initialized randomly -> Only during initialization!
• initialization has to take place every MC-step, not once in the beginning

@stheid ideas for structure?

[Webbah]

• [x] add delay (1 timestep) between controller and env
• [x] use unbalanced loads

[Webbah]

Implemented in #59