MCP-0017: FMU_Import - Githubissues

modelica-trac-importer commented 6 years ago

Modified by dietmarw on 9 Jun 2015 12:41 UTC This is the ticket for MCP 0017:

The primary goal is to important an FMU 2.0 ModelExchange or CoSimulation in a fully portable way into Modelica. Basically, the FMU is a Modelica ExternalObject (Modelica 3.3 section 12.9.7) and the FMU functions are Modelica functions operating on this ExternalObject. An ExternalObject has two required functions: “constructor” + “destructor”, that are called by the simulation environment when appropriate. It is proposed to add three additional optional functions that an ExternalObject can have. These functions are also useful for non-FMU external objects.

Note, currently, import of FMUs is tool specific, e.g. due to function fmi2CompletedIntegratorStep. Initially, this was dicussed in ticket #1626.

Document location

https://svn.modelica.org/projects/MCP/MAinternal/MCP-0017_FMU_Import/

Reported by otter on 9 Jun 2015 12:24 UTC This is the ticket for MCP 0017:

The primary goal is to important an FMU 2.0 ModelExchange or CoSimulation in a fully portable way into Modelica. Basically, the FMU is a Modelica ExternalObject (Modelica 3.3 section 12.9.7) and the FMU functions are Modelica functions operating on this ExternalObject. An ExternalObject has two required functions: “constructor” + “destructor”, that are called by the simulation environment when appropriate. It is proposed to add three additional optional functions that an ExternalObject can have. These functions are also useful for non-FMU external objects.

Note, currently, import of FMUs is tool specific, e.g. due to function fmi2CompletedIntegratorStep. Initially, this was dicussed in ticket #1626.

Migrated-From: https://trac.modelica.org/Modelica/ticket/1727

modelica-trac-importer commented 6 years ago

Comment by dietmarw on 9 Jun 2015 12:41 UTC Added document location to description.

modelica-trac-importer commented 6 years ago

Comment by hansolsson on 14 Sep 2016 10:35 UTC Hans: Would prefer to merge enterEventMode(obj) and enterContinuousTimeMode(obj) to enterMode(obj, ...); - could then possibly add more transitions in the state-diagram to this function. Henrik: There are several states in the state machine for FMU, and multiple transitions - in some cases the transition-functions are not called. Hans: Should ensure that enterMode(obj, ...) is called consistently. Should it have both previous and current mode as input? Henrik: FMU should take of previous mode.

Will have to ensure that this works for all transitions in state-diagram. Convenient to have redundant enterMode(obj, ...) defined as no-op.

Is there always a clear place for calling this function in a simulation environment? -- Resource location: Two issues:

Moving packages should move resources!

Hans: Already part of Dymola/3D Experience platform move/copy operations, similarly as references, or isn't it? Could be confusion by LibraryDirectory - that is handled this way.

Henrik: Should allow relative paths as well in loadResource. Note: Normal structure is:

M 
 - package.mo
 - A 
   - B.mo
   - package.mo
 - Resources - even for M.A.B

Seems like tool-issue about moving/copying classes.

Resources inside an FMU. The idea is that loadResource ensures that a table-file can be referenced, a FMU built referencing this table-file, sent to someone else and the resource is found when running the FMU. Internally this is handled by having the table-file in the FMU, and using a reference returns this - based on fmiLocation sent to instantiate model.

fmi2SetupExperiment: Hans: We need this an external function of the FMU, but having consistent argument with the simulation seems less urgent, we could expose them as parameters of the imported FMU and require users to set them for the imported (and that could be that different from simulation setup). Two possibilities: access the information in Modelica and automatically call this function in wrapper, or automatically call this function for external objects - if provided. Gerd: There are some cases where start-time is used currently in Modelica. Hans: Risk of misuse in models; will we need fake stop-time in simulation for debugging?

Nominal attribute seems less urgent - since work-around exist and nominal is not *that* necessary. Value reference problem seems urgent: started discussion

modelica-trac-importer commented 6 years ago

Comment by hansolsson on 14 Sep 2016 12:07 UTC The solution for the unsigned int as value reference in Dymola and SimulationX is to treat it as integer - and let it overflow (more or less controlled). The change of the external interface is not that needed.

In SystemModeler additional integer types are available in the external environment (as we interpreted Henrik). -- Additional changes are needed for portable efficient import of FMUs (especially setting and getting variables allowing for algebraic loops over the FMU). It is not clear if the non-linear predictive control use case will benefit from this - or if the automatic modeChanges will be a problem, and the wrapper needs to be adapted.

Both for FMU and other numerical use case it is important that completedIntegratorStep specify that the directly preceding calls are for the state at the end-point of the step (that would not necessarily be the case for all integrators).

For maximum of a signal it requires the derivative of the input, whereas one goal was to avoid that requirement.

Using the ModelicaError workaround for FMUs seem dangerous: this will involve longjmp between different DLLs - and it is not clear that the FMU-interface allows functions to not return (the error logger in FMU is only supposed to log the error). And for external objects (in DLL) you can assume the first call constructs the external object, and the last one deletes the external objects.

Using the relative tolerance in other places is not a good idea - since the error might cause chattering, since the crossing functions will have an error of the order 1e-4. Thus this is actually an argument against providing that information to models. -- How to proceed:

Allow these optional functions in external objects (modeChange and completedIntegratorStep - where the object should be in the state corresponding to the completed step).
Document the behavior of loadResource.
Test-implement - and see what is missing

Hans: What to do if there is a state event during a step, should completedIntegratorStep first be called at the end-point of the original step? Gerd: At the point found by the event-locator - i.e. the point where the event will be handled, and the same time-point as the starting point of the next step.

modelica-trac-importer commented 6 years ago

Comment by hansolsson on 14 Sep 2016 12:10 UTC Unclear who will volunteer to do it. Stefan V doesn't work on FMUs, neither Gerd nor Hans think they have time.

modelica-trac-importer commented 6 years ago

Modified by stefanv on 25 Oct 2017 13:32 UTC

GallLeo commented 5 years ago

We see more and more Modelica users store their FMU wrappers in Modelica packages. Those Modelica packages are then not Modelica-compliant anymore.

I'm courious why this MCP had high priority in the beginning and then came to a full stop. The last comment indicates, that nobody had time to work on it? Or is it because it's impossible to solve satisfactory for all tools?

chrbertsch commented 5 years ago

I would appriciate this also w.r.t. to the overall goal that the MA standards should be compatible with each other and together allow seamless workflows.

GallLeo commented 4 years ago

Is this MCP scheduled for Modelica 3.5?

HansOlsson commented 4 years ago

Is this MCP scheduled for Modelica 3.5?

Realistically (and unfortunately) not, since there hasn't been enough work on it.

chrbertsch commented 4 years ago

As FMI 3.0 is now in alpha phase with a stable API, it would be very import to priorize this topic now. (For FMI 2.0, every tool has already a solution, some better, some worse ...)

But every Modelica Tool vendor will have to spend effort to implement FMI 3.0 support anyway, especially for the new co-simulation types HCS, SCS but also for the other changes and new features of FMI 3.0 ...

Having portable FMI 3.0 support in Modelica Tools would bring the MA's goal of coordinated standards forward and thus help both Modelica tools and the adoption of FMI 3.0. Starting this effort in the FMI 3.0 alpha phase could also provide important feedback to the FMI 3.0 development that still could be incorporated in the FMI 3.0 release.

@ MA Language group: currently the focus was on FMI 2.0, right? How far is the MCP for FMI2.0 away from finalization? Could you please discuss to extend this for FMI 3.0 and estimate the effort?

@ MA Board : could you please discuss this, as this is also a strategically important issue for MA. How can we accelerate this work, which is open for many years and now becomes urgent for FMI 3.0? Could part of the work be supported by someone working on parts of this on a payed contract (following requirements defined by the Modelica Language Project)?

@ Tool vendors: Are you willing to join this effort?

henrikt-ma commented 4 years ago

I've been working on FMI import for SystemModeler, and as a long term goal, I totally agree that true Modelica support for full FMI import is important for keeping the MA standards coordinated. However, in my mind, solving this problem for FMI 2 has proven so difficult that accomplishing this goal by means of small modular additions to the Modelica language seems almost impossible in the foreseeable future. Regarding FMI 3.0, I don't know enough to determine what sort of impact it may have on the timeline.

I believe we might be able to come up with some modular additions to the Modelica language that would make a portable FMI import sort of do the right thing, but I'm afraid it would deliver unacceptable simulation results compared to the tool-specific integration methods used today. As long as simulation results are unacceptable, however, I don't see much use for such a portable but half-baked FMI import.

HansOlsson commented 1 year ago

Design meeting:

Propose that it is worked on by a working group for the MCP ( @HansOlsson in a few months):
- Should specify which functions are needed, and show complete solution with them (for FMI 2) - also show that good performance (with loops for ME etc)
- Too hard to do without that interest
- Once ready, others can check that it works in other tools
Should also consider licensing - so that users don't create FMUs to protect IP instead of just sending models: both standardized encryption and just including license files

Standardizing the overall interface for FMU in the Modelica specification seems too complex (and too many variants within one tool), so easier to move imported FMU to another tool.

t-sommer commented 1 year ago

FWIW, FMPy has an experimental, tool independent FMU import for Modelica (tested with Dymola and OpenModelica) that might serve as a basis for this discussion. It also comes with a set of imported demo model from the Reference FMUs for FMI 2.0, 3.0, Co-Simulation, and Model Exchange. The Modelica library is part of the FMPy distribution under <python-packages>/fmpy/modelica/FMI/package.mo.

The easiest way to import your own FMU is to load it in the FMPy GUI and select Tools > Import to Modelica....

Note that Python is only used to generate the Modelica code. The runtime library is only static Modelica code with external C functions that are pre-compiled for Mac, Linux, and Windows.

chrbertsch commented 1 year ago

FWIW, FMPy has an experimental, tool independent FMU import for Modelica (tested with Dymola and OpenModelica) that might serve as a basis for this discussion. It also comes with a set of imported demo model from the Reference FMUs for FMI 2.0, 3.0, Co-Simulation, and Model Exchange. The Modelica library is part of the FMPy distribution under <python-packages>/fmpy/modelica/FMI/package.mo.

The easiest way to import your own FMU is to load it in the FMPy GUI and select Tools > Import to Modelica....

Note that Python is only used to generate the Modelica code. The runtime library is only static Modelica code with external C functions that are pre-compiled for Mac, Linux, and Windows.

FMI-Design-Webmeeting: Is there feedback from tool vendors? Torsten: First tests with Dymola, OpenModelica, Modelon.Impact have been successful Kaska: I will take a look. Are there still issues with completed integration step? Torsten: yes this is open; I could provide a solution for Dymola; tool vendors could fix this on their own (this would be outside of the reference implementation). Finding a standardized solution for this would be good. This is only an issue for ME, which is not so often used.

Maplesoft-fmigroup commented 1 year ago

@t-sommer I've been able to import the modelica package into MapleSim; however, it looks like the zipped archive of the depot is missing ModelicaFMI binaries in FMI/Resources/Library/platform directories so we can't simulate any models. Also, is this intended as a stand-alone FMU import or can the FMU be connected to other FMUs and modelica components? If the latter then there's an issue with using the 'initial algorithm' code as-is in that, due to discrete iteration in models with algebraic loops, the initial algorithm may have to run multiple times. So we may have to update inputs while still in initialization mode while the current initial algorithm exits initialization mode at the end.

t-sommer commented 1 year ago

The Python wheel contains the pre-built ModelicaFMI.* libraries for darwin64, linux64, win32, and win64 that are part of the FMPy installation (see my comment above).

If you're only interested in the FMI Modelica library you can download and extract the wheel (e.g. with 7Zip) and load FMPy-0.3.15-py3-none-any.whl/fmpy/modelica/package.mo.

If you want to build the binaries from source you have to follow the steps in azure-pipelines.yml for your platform.

HansOlsson commented 1 year ago

I couldn't find ready-made imported Modelica, but having worked on FMI-import I see several issues:

Support for completed integrator step for Model-Exchange FMUs (for FMUs that require it). That has always been tool-specific hack in Dymola (as Modelica doesn't have the capability), and I don't see that FMPy handles it.
Using dependency information (especially for Model-Exchange FMUs) to efficiently handle systems of equations involving Model-Exchange FMUs.
The two comments above are sort of a generalization of one concern from MapleSim (that focused on that problem during initialization).
I haven't investigated the new FMI 3 features, but I see that they are difficult to support efficiently.

casella commented 1 year ago

The issue of FMU import in OpenModelica is discussed in OpenModelica/OpenModelica#5345 (spoiler alert: very, very long discussion).

As we currently understand

not all FMUs can be imported in Modelica, because the semantics of FMUs is procedural, and some features cannot be expressed in a declarative, equation-based way (e.g., discard and re-do step)
there is clearly a class of FMUs that can indeed be imported in Modelica without issues
this class contains purely continuous time DAE models. Not sure what its actual boundaries are

From my point of view, it would be nice if we could determine a set of sufficient (if not necessary) conditions for an FMU to be importable in Modelica.

I would add one extra issue to @HansOlsson's list

FMUs are easily imported in Modelica as algorithms. However, reinit() is not allowed in there.

modelica / ModelicaSpecification

MCP-0017: FMU_Import #1727

Document location