search

modelica / ModelicaSpecification

Specification of the Modelica Language
https://specification.modelica.org
Creative Commons Attribution Share Alike 4.0 International
96 stars 41 forks source link

Introduce external object save/restore #1950

Open modelica-trac-importer opened 5 years ago

modelica-trac-importer commented 5 years ago

Reported by sjoelund.se on 3 Mar 2016 16:34 UTC Related to https://github.com/modelica/ModelicaStandardLibrary/issues/1899 https://github.com/modelica/ModelicaStandardLibrary/issues/1941, where various C-code changes are proposed to make it possible to restart a simulation without executing initial equations (only external object constructors).

I propose to instead introduce save/restore functions to external objects to make support for this explicit in Modelica so that everything works without requiring too much thought to go into designing the external objects around this feature. It would then be much simpler to map the Modelica simulation state to FMI save/restore. And if an external object did not implement this, simulation cannot be (reliably) resumed.

class MyExtObj
  function constructor
    // As before. But NOT called when resuming simulation!
  end constructor;
  function destructor
    // As before.
  end destructor;
  function save
    input MyExtObj obj;
    output XXX data;
  external "C" ...;
  end save;
  function restore
    // Is called with the saved data instead of the constructor when resuming
    input XXX data;
    output MyExtObj obj;
  external "C" ...;
  end restore;

The problem is deciding what XXX should be. I thought about Integer[:] at first, but there is no way to specify the size of the array from the outside (but would be possible to use by also introducing a function returning the size required to serialize the object). ModelicaAllocateString could work to just allocate data (but high bit must not be set and no NULL elements in the string is annoying). It could of course be that we just return a filename containing the serialized object.

Migrated-From: https://trac.modelica.org/Modelica/ticket/1950

modelica-trac-importer commented 5 years ago

Comment by sjoelund.se on 3 Mar 2016 16:43 UTC I forgot to mention it in the description: the feature would be optional for a Modelica tool to implement (which is useful if the tool does not support resuming simulation anyway).

modelica-trac-importer commented 5 years ago

Modified by beutlich on 3 Mar 2016 18:05 UTC

modelica-trac-importer commented 5 years ago

Comment by hansolsson on 4 Mar 2016 09:17 UTC I believe this is an important extension - especially for FMUs. The simplest case is just an Integrator as co-simulation FMU, we need to save and restore that sate. But it isn't needed for the tables (as far as I know, and unless we add new features like hysteresis), since all the information for a restarted simulation is present in the original model. -- Returning a file-name where the file contains the serialized contents would be messy since the ExternalObject would have to open a temporary file, and each external object would need routines for that etc.

But there is a simple solution if we want to use files: Both save and restore gets the file-name as input - and save/restore will just use fopen() - fputc()/fgetc() - fclose.

modelica-trac-importer commented 5 years ago

Comment by hansolsson on 4 Mar 2016 17:42 UTC Before I forget, just a minor question: is save-restore a good pair of names, or would save-load, store-restore, or save-resume be better?

modelica-trac-importer commented 5 years ago

Comment by stefanv on 4 Mar 2016 17:47 UTC To me, it's a toss-up between save/restore and save/resume. I don't like save/load because they imply saving and loading something static (e.g. a program or document), instead of dynamic (e.g. the state of a running simulation).

Save/restore is probably a slightly better choice than save/resume, because resume implies that the simulation will immediately continue, whereas restore only implies that the state will be restored, so that it can continue (when everything else that needs to be restored has been restored).

modelica-trac-importer commented 5 years ago

Comment by choeger on 5 Mar 2016 12:34 UTC I do not think it would be very wise to expose the serialized data to the user directly (i.e. have an array of integers, a file or something like this as input). Instead I would leave the serialized data abstract and pass the instance name of the component to restore. Think of it as a key to lookup. In order to deal with this, any implementation has to provide proper C-routines for looking up and storing serialized data. Any external object might use these C-routines and hence all the ugly memory access and state manipulation stays where it belongs, under the hood in C-land. The connection between the serialized data and the current simulation can then be left tool specific (i.e. a database, a file or whatever).

modelica-trac-importer commented 5 years ago

Comment by hansolsson on 6 Mar 2016 19:26 UTC To be genuine useful I think we should aim for the possibility of restoring the object to the same state - including the cases where the external object is in a different model.

Use-cases would be:

Thus we should avoid using the following:

modelica-trac-importer commented 5 years ago

Comment by hansolsson on 6 Mar 2016 19:33 UTC Replying to [comment:6 choeger]:

I do not think it would be very wise to expose the serialized data to the user directly (i.e. have an array of integers, a file or something like this as input).

The entire functionality should be hidden from the end-user; similarly as the constructor as destructor.

The external-object creator writes the functions (in a tool-independent way)- the user just write TheirExternalObject a(...); - and then the tool calls everything when needed. By allowing the state to saved it is possible to reload an old simulation - and investigate alternative scenarios - or just continuing the simulation.

modelica-trac-importer commented 5 years ago

Comment by choeger on 7 Mar 2016 08:16 UTC Replying to [comment:8 hansolsson]:

The external-object creator writes the functions (in a tool-independent way)- the user just write TheirExternalObject a(...); - and then the tool calls everything when needed.

In the proposal above, the user uses 

TheirExternalObject a(serializedData);

This also allows:

serializedData[42] := 0;
TheirExternalObject a(serializedData);

I do not think that it would be wise to allow that.

modelica-trac-importer commented 5 years ago

Comment by sjoelund.se on 7 Mar 2016 09:09 UTC To be clear, save and restore would be forbidden to call directly by the user (just like the constructor and destructor are forbidden to call directly). The simulation tool would automatically call these functions when it tried to save/restore the simulation state.

modelica-trac-importer commented 5 years ago

Modified by beutlich on 5 Aug 2016 08:41 UTC

modelica-trac-importer commented 5 years ago

Comment by hansolsson on 12 Sep 2016 15:52 UTC Agreement that seems like a good functionality.

One issue is that not calling "constructor" is that constant data needs to be serialized, another option would be to first call "constructor" and then "restore" - so that only the non-constant data needs to be stored.

Another option would be that restore first have constructor-inputs, followed by serialized data. -- Henrik: The use of this feature will be tool-specific (since resuming simulation isn't specified). Should we have some Modelica-useable functionality for this as well? E.g. store external state in specific file/string and then resume to a specific state. E.g. call it "serialize/deserialize" - and have that user-callable (and also use internally for this purpose). What shall the state be? String (problem with character encoding and especially NUL - or use base64 and don't care about memory use), vector - problem with returning length - and how to return from C. Seems that base64 is the best - but can be handled internally in the external object, if we just specify that it is a normal Modelica String.

Need an MCP, Martin S may start on that.

modelica-trac-importer commented 5 years ago

Comment by hansolsson on 22 Mar 2018 08:34 UTC Language group: Some discussion about files vs. strings or C-like (char* and length) for storing data. Not clear that string in Modelica is ideal for this - we might have another function for this; there was something added in FMI for binary blobs - could reuse.

Another option would be to have another external object with functions for this - and send this as argument.

There were multi-threading concerns - but seems that they should not change.

Martin S. does not think he has time. Gerd: might go together with other extensions of external objects.

KarlWernersson commented 3 years ago

Hi I will try to reawake this discussion.

In FMI we can serialize the internal state in to a Byte array represented by char[] and size_t

It has two levels,

typedef void* fmi2FMUState;
fmi2Status fmi2GetFMUstate (fmi2Component c, fmi2FMUstate* FMUstate);
fmi2Status fmi2SetFMUstate (fmi2Component c, fmi2FMUstate FMUstate);
fmi2Status fmi2FreeFMUstate(fmi2Component c, fmi2FMUstate* FMUstate);

These functions allocates/frees retrieves and restore the FMUs state of the FMU in the shape of a void pointer that can be kept in the memory.

This FMUstate can then be passed or retrieved to the functions

typedef char fmi2Byte;
fmi2Status fmi2SerializedFMUstateSize(fmi2Component c, fmi2FMUstate FMUstate, size_t *size);
fmi2Status fmi2SerializeFMUstate (fmi2Component c, fmi2FMUstate FMUstate, fmi2Byte serializedState[], size_t size);
fmi2Status fmi2DeSerializeFMUstate (fmi2Component c, const fmi2Byte serializedState[], size_t size, fmi2FMUstate* FMUstate);

These functions retrieves the size of the FMUstate, and can sterilize the state onto a Byte[] or use a give byte vector to create an FMUstate that can be used to reset the FMU.

So for me as an fmuExporter of a modelica model containing external objects. What I need is sufficient information and data form the external object that its internal state can be bijective mapped to a char[n] +size_t n.

Using files for this would be very messy when the model is contained in an FMU that shall support serialization of FMU state or set/get so if you want to support the file IO for this functionality, sperate it from retrieving the external object state, this could then easily be passed on to the fIO functionality.

It is unclear for me if the intermediate FMUstate is relevant for modelica models, or you only want to give the serializedState. One use case if of course to better support get/setFMUstate wihhout having to serilize the internal state of the external object, but I don’t see that as crucial.

I am happy with a char [n] and size_t n, but as long as you can guarantee that he way the information can be mapped to that it should not be a problem.

KarlWernersson commented 3 years ago

Not all external object would need this functionality, it would be nice to be able to distinguish the objects that don't need the functionality form the objects where the designer has not implemented it.

beutlich commented 3 years ago

See also https://github.com/modelica/ModelicaStandardLibrary/issues/3759.