using fmu as a state machine: initialize flag in simulate_fmu

[philippGraf]

Hello,

I am coming back to https://github.com/CATIA-Systems/FMPy/issues/277 I have tested the approach using the fmu_instance argument of the simulate function. To get a state machine from the fmu as needed for using it in an openai/gym context, the last result values had to be stored in a python class. This stored state has then to be be put as start_values in the next simulation. This approach led to a stable sate machine, nevertheless the computational performance was poor on a non trivial example. The state machine proposed in https://github.com/CATIA-Systems/FMPy/issues/277 performed about 2.4 times faster then the one using the start_values argument to propagate the state. In the context of machine learning this is a serious drawback. The reason for this performance drawback is found in the unneccessary initialization of the fmu. Moreover, it seems counterintiutive not to use the fmu itself as state machine, but to wrap a python class around it keeping the state. With this in mind I propose the following minor change of the simulate_fmu function:

https://github.com/ConsolinnoEnergy/FMPy/blob/master/fmpy/simulation.py

By adding the logical flag "initialize" the initialization method of the fmu may be skipped in order to keep the state of the instantiated fmu. The fmu is initialized once and simulations may then be performed step by step.

The neccessary changes in code are minimal and the usage of the fmu as a state machine is revealed in simple and natural way. I guess, this idea follows the spirit of the project by providing efficient access to fmu in a minimal way.

[t-sommer]

To get a state machine from the fmu as needed for using it in an openai/gym context, the last result values had to be stored in a python class. This stored state has then to be be put as start_values in the next simulation.

This approach does not work in general as the FMU might not expose all internal states as variables. You have to use the fmi2{Get/Set}FMUstate() API instead.

[t-sommer]

So, is your goal to implement gym.Env with an FMU instance inside?

[philippGraf]

Concerning the states: When we build up fmu's, we define what variables of the modell determine an observation before hand. These are then variables with causality: output. So I think from the point of view of this workflow, all neccessary states/ observations are accessible. But you are right, there should be some kind of documentation for the user somewhere, how to prepare fmu's for proper use with gym. Nevertheless, i do not see this documentation as a direct issue of the simulate_fmu function.

[philippGraf]

And you are correct, that is exactly the goal: Pretrain AI-Models (Reinforcement-Learning) on fmu-twins of physical/technical systems with an openai/gym-interface and then release them into the world. In this manner we may separate the implementation of the physical (modelica) from the implementation of the ai-learning. This improves the development of such ai-applications.

[t-sommer]

I've tried to run the GoLeftEnv example from https://stable-baselines.readthedocs.io/en/master/guide/custom_env.html in a local Conda environment but didn't succeed due to dependency conflicts. Gym (or one of its dependencies) seems to rely on some very old versions of libraries.

[philippGraf]

try to pip install the package from the conda terminal ..... : https://github.com/openai/gym/issues/809 But I do not have any conda experience ...

[philippGraf]

by the way have you had a look at: https://github.com/ConsolinnoEnergy/FMPy/blob/master/fmpy/examples/efficient_loops_with_state.py ? No gym needed here. I also used it as a test (just like you did) and i guess it serves good for illustration of the new mechanics of the function. The gym specific impemetations are now in the gym-branch: https://github.com/ConsolinnoEnergy/FMPy/tree/gym-branch/ In https://github.com/ConsolinnoEnergy/FMPy/tree/gym-branch/fmpy/examples you'll find the non trivial example and the interface is in https://github.com/ConsolinnoEnergy/FMPy/blob/gym-branch/fmpy/gym_interface.py

[t-sommer]

by the way have you had a look at: https://github.com/ConsolinnoEnergy/FMPy/blob/master/fmpy/examples/efficient_loops_with_state.py

I'm a bit confused. Your code does not use the serialization (get/set FMUstate) API. See https://github.com/CATIA-Systems/FMPy/blob/master/tests/test_serialize_fmu_state.py for an example.

[philippGraf]

ok, now it's getting interesting... no it does not, as the modified simulate_fmu function does not reset (no initialize is called) the state of the fmu in the beginning, when initialize = False. In this way the fmu keeps the state itself. The skipping of the initialize method gives the performance boost, as the simulation (do_step) simply continues.

[t-sommer]

Again: the only way to restore the state of an FMU is to use the serialization API. Other methods might work "accidently" but I would not recommend it.

[t-sommer]

Please open a new issue on your project to continue the discussion.