search

ThummeTo / FMI.jl

FMI.jl is a free-to-use software library for the Julia programming language which integrates FMI (fmi-standard.org): load or create, parameterize, differentiate and simulate FMUs seamlessly inside the Julia programming language!
MIT License
84 stars 24 forks source link

fmiReload and fmiLoad not thread-save #186

Open AnHeuermann opened 1 year ago

AnHeuermann commented 1 year ago

Issue

I want to simulate a number of FMUs in parallel with FMI.jl and reuse the already loaded FMUs for multiple runs, but encounter an error in FMIImport where cd is used to change the directory while running in parallel.

ERROR: LoadError: IOError: pwd(): no such file or directory (ENOENT)
Stacktrace:
  [1] uv_error
    @ ./libuv.jl:97 [inlined]
  [2] pwd()
    @ Base.Filesystem ./file.jl:63
  [3] loadBinary(fmu::FMICore.FMU2)
    @ FMIImport ~/.julia/packages/FMIImport/Zfd3H/src/FMI2/ext.jl:246
  [4] fmi2Load(pathToFMU::String; unpackPath::String, type::Nothing, cleanup::Bool, logLevel::UInt32)
    @ FMIImport ~/.julia/packages/FMIImport/Zfd3H/src/FMI2/ext.jl:240
  [5] #fmiLoad#168
    @ ~/.julia/packages/FMI/PEsZa/src/FMI.jl:616 [inlined]

I believe the issue is the usage of cd in FMIImport ext.jl#L245-L252 Is cd needed here? But I guess there are other problems with using e.g. ccall in parallel, see https://docs.julialang.org/en/v1/manual/multi-threading/#@threadcall.

If it's not possible to use fmiLoad and fmiReload in parallel, maybe add a check to them to test if they are run inside a parallel region?

How to reproduce

fmuArray = [FMI.fmiLoad(fmuPath) for _ in 1:10]
locks = [ReentrantLock() for _ in 1:10]
Threads.@threads for i in 1:100
    idx = ((i-1)%10+1)
    lock(locks[idx]) do
      fmu = fmuArray[idx]
      # Do something with the FMU
      FMI.fmiInstantiate!(fmu)
      FMI.fmiSetupExperiment(fmu)
      FMI.fmiEnterInitializationMode(fmu)
      FMI.fmiExitInitializationMode(fmu)

     # Reload FMU
      FMI.fmiReload(fmu)
    end
end

You can get similar results with

Threads.@threads for i in 1:100
         lastDirectory = pwd()
         mkdir("temp_$i")
         cd("temp_$i")
         sleep(5)
         cd(lastDirectory)
end
ThummeTo commented 1 year ago

We will check this.

PS: In the current release, you don't need to load the FMU multiple times for multithreading. Loading it once and use fmiSimulate in a Thread-Loop is sufficient (one fmi2Instance is allocated per Thread, all Threads share the same FMU/DLL). But the tutorial "multiprocessing" needs an update (multiple FMUs are loaded there).

AnHeuermann commented 1 year ago

Any news on this issue? My scripts are generating a bunch of segmentation faults when running on a single FMU in multiple threads and FMI.jl is reporting errors when I re-use a FMU on a different thread. So I can't let my scripts run over night, because they'll break most of the time.

ERROR: LoadError: TaskFailedException

    nested task error: AssertionError: ["No FMU instance allocated (in current thread with ID `18`), have you already called fmiInstantiate?"]
ThummeTo commented 1 year ago

Hm... I am using multithreading on a single FMU quite often. There is no need to load/unload the FMU in this operation mode multiple times. I do this not only for simulation, but also gradinet determination in NeuralFMUs. You should be able to load the FMU a single time (on the main thread), simulate it on different threads (monte carlo style) and being able to unload it after all threads finished (again on the main thread).

I am working on an update for the multithreading tutorial. But there where multiple other ToDos the last months (speed, sensitivities, ...)

ThummeTo commented 1 year ago

So a typical workflow might look like:

(1) load FMU (its a dynamic library, so its ok to just load it once) (2) do things with multiple instances of the FMU (FMI.jl allocates a new instance for every simulation by default) in parallel, but dont load/unload/reload the FMU (no need to do that) (3) finally - after every thread finished - unload the FMU at program end

AnHeuermann commented 1 year ago

I think I found my issue. I was

My mwe looks something like this:

FMU from

model HelloWorld "Simple hello world example"
  Real x(start=1, fixed=true);
  Real y;
  Real z;
  parameter Real a = -0.5;
equation
  der(x) = a*x;
  y = sin(x);
  z = 2*abs(x);
end HelloWorld;

generated with OpenModelica:

loadFile("helloWorld.mo"); getErrorString();
buildModelFMU(HelloWorld); getErrorString();
import FMI
import FMIImport

ENV["JULIA_DEBUG"] = "FMI,FMIIMport,FMICore"

function runAll()
  fmuPath = "fmu/HelloWorld.fmu"
  inputVars = ["x"]
  outputVars = ["y", "z"]

  nInputs = length(inputVars)
  nOutputs = length(outputVars)
  nVars = nInputs + nOutputs

  fmu = FMI.fmiLoad(fmuPath)

  Threads.@threads for _ in 1:Threads.nthreads()*10
    @info "Thread $(Threads.threadid()) starting!"
    comp = FMI.fmiInstantiate!(fmu; loggingOn = false, externalCallbacks=false)
    FMI.fmiSetupExperiment(comp)
    FMI.fmiEnterInitializationMode(comp)
    FMI.fmiExitInitializationMode(comp)

    # Set random data
    row = Array{Float64}(undef, nVars)
    row_vr = FMI.fmiStringToValueReference(fmu.modelDescription, vcat(inputVars,outputVars))

    for _ in 1:10
      row[1:nInputs] = rand(nInputs)

      FMIImport.fmi2SetReal(comp, row_vr, row)
      row[nInputs+1:end] .= FMIImport.fmi2GetReal(comp, row_vr[nInputs+1:end])
    end

    #FMI.fmiFreeInstance!(comp)
    @info "Thread $(Threads.threadid()) finished!"
  end

  FMI.fmiUnload(fmu)
end

Output:

julia> runAll()
[ Info: Thread 2 starting!
[ Info: Thread 13 starting!
[ Info: Thread 19 starting!
[ Info: Thread 14 starting!
[ Info: Thread 3 starting!
[ Info: Thread 11 starting!
[ Info: Thread 5 starting!
[ Info: Thread 9 starting!
[ Info: Thread 11 starting!
[ Info: Thread 3 starting!
[ Info: Thread 8 starting!
[ Info: Thread 18 starting!
[ Info: Thread 16 starting!
[ Info: Thread 19 starting!
[ Info: Thread 10 starting!
[ Info: Thread 1 starting!
[ Info: Thread 4 starting!
[ Info: Thread 14 starting!
[ Info: Thread 13 starting!
[ Info: Thread 6 starting!
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb49ee39ae0, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb59c387290
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb59c387290, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb4a15b0580, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb594257a50
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb59c387290) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb594257a50, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb59c387290) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb594257a50) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb59c387290, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.007785949875737441, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb4a14a5150, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb56c31e9c0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb594257a50) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb594257a50, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.5500224536739118, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb59c387290, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.007785871210611238, 0.015571899751474882]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb56c31e9c0, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb59c387290) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb56c31e9c0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb594257a50, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.5227063711065167, 1.1000449073478236]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
[ Info: Thread 2 finished!
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb56c31e9c0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb594257a50) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb499d55420, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb55444f3b0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
[ Info: Thread 3 finished!
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb56c31e9c0, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.8363120420636442, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb55444f3b0, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb56c31e9c0, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.7421764867727053, 1.6726240841272884]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb55444f3b0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb4a1345750, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb58c44f120
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb55444f3b0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb56c31e9c0) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb58c44f120, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
[ Info: Thread 12 finished!
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb58c44f120) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb55444f3b0, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.8703188848113075, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb58c44f120) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb49db567a0, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb57c2df520
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb55444f3b0, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.7645345235525051, 1.740637769622615]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb57c2df520, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb55444f3b0) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
[ Info: Thread 15 finished!
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb58c44f120, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.3752589544361712, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb57c2df520) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb58c44f120, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.36651347587929817, 0.7505179088723424]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb57c2df520) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb58c44f120) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
[ Info: Thread 6 finished!
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb57c2df520, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.0542789737565329, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb4a13993f0, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb5441c26e0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb57c2df520, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.054252324833830703, 0.1085579475130658]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb5441c26e0, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb57c2df520) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb5441c26e0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb4a1225bd0, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb5340b6270
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
[ Info: Thread 9 finished!
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb5441c26e0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb5340b6270, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb5441c26e0, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.5863938555148537, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb5340b6270) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb5441c26e0, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.5533609197270968, 1.1727877110297074]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb4a1569330, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb5503802b0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb5441c26e0) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb5503802b0, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb5340b6270) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb4a14a5270, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb55c235de0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
[ Info: Thread 13 finished!
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb5503802b0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb55c235de0, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb5340b6270, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.7396688106025051, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb5503802b0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb55c235de0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb5503802b0, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.17655822007580613, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb55c235de0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb5503802b0, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.17564234626318317, 0.35311644015161225]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb55c235de0, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.7221951008339035, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb5503802b0) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
┌ Debug: fmi2Instantiate(instanceName: Ptr{UInt8} @0x00007fb4b12f2038, fmuType: 0, fmuGUID: Ptr{UInt8} @0x00007fb5afcc1cf8, fmuResourceLocation: Ptr{UInt8} @0x00007fb4b0847bd8, functions: Ptr{FMICore.fmi2CallbackFunctions} @0x00007fb49db56770, visible: 0, loggingOn: 0) → Ptr{Nothing} @0x00007fb594374420
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:25
[ Info: Thread 17 finished!
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb5340b6270, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.6740433016954693, 1.4793376212050102]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178

[11758] signal (11.1): Segmentation fault
in expression starting at REPL[2]:1
omc_util_get_pool_state at /tmp/fmijl_2A606N/HelloWorld/binaries/linux64/HelloWorld.so (unknown line)
updateIfNeeded at /tmp/fmijl_2A606N/HelloWorld/binaries/linux64/HelloWorld.so (unknown line)
fmi2GetReal at /tmp/fmijl_2A606N/HelloWorld/binaries/linux64/HelloWorld.so (unknown line)
fmi2GetReal! at /home/USER/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:174
fmi2GetReal! at /home/USER/.julia/packages/FMIImport/hLBHK/src/FMI2/c.jl:558 [inlined]
fmi2GetReal at /home/USER/.julia/packages/FMIImport/hLBHK/src/FMI2/int.jl:143
unknown function (ip: 0x7fb4a9936b7f)
┌ Debug: fmi2SetupExperiment(c: Ptr{Nothing} @0x00007fb594374420, toleranceDefined: 0, tolerance: 0.0, startTime: 0.0, stopTimeDefined: 0, stopTime: 0.0) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:103
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb5340b6270) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
[ Info: Thread 19 finished!
┌ Debug: fmi2EnterInitializationMode(c: Ptr{Nothing} @0x00007fb594374420) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:118
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
┌ Debug: fmi2ExitInitializationMode(c: Ptr{Nothing} @0x00007fb594374420) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:133
#fmi2GetReal#10 at /home/USER/.julia/packages/FMI/1VUBe/src/FMI2/comp_wraps.jl:107
fmi2GetReal at /home/USER/.julia/packages/FMI/1VUBe/src/FMI2/comp_wraps.jl:106 [inlined]
macro expansion at /home/USER/workspace/Testitesttest/FMI-multithread/runAllTheFMUs.jl:34 [inlined]
#6#threadsfor_fun#3 at ./threadingconstructs.jl:163
#6#threadsfor_fun at ./threadingconstructs.jl:130 [inlined]
#1 at ./threadingconstructs.jl:108
unknown function (ip: 0x7fb4a992aa0f)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
┌ Debug: fmi2SetReal(c: Ptr{Nothing} @0x00007fb594374420, vr: UInt32[0x00000000, 0x00000002, 0x00000003], nvr: 3, value: [0.6784114589340956, 0.0, 0.0])
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:193
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
start_task at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/task.c:1092
Allocations: 61803428 (Pool: 61766991; Big: 36437); GC: 98
┌ Debug: fmi2GetReal(c: Ptr{Nothing} @0x00007fb594374420, vr: UInt32[0x00000002, 0x00000003], nvr: 2, value: [0.6275570249747953, 1.3568229178681912]) → 0
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:178
┌ Debug: fmi2FreeInstance(c: Ptr{Nothing} @0x00007fb594374420) → [nothing]
└ @ FMICore ~/.julia/packages/FMICore/JhS6T/src/FMI2/cfunc.jl:41
[ Info: Thread 3 finished!
[1]    11758 segmentation fault  julia --threads=auto
ThummeTo commented 1 year ago

Is this error FMU-specific? If no, can you use one public available (like from the modelica refernce FMUs or the FMIZoo.jl). This would be much easier to reproduce, because I don't have OM installed.

AnHeuermann commented 1 year ago

I couldn't reproduce the same issue, but with 2.0 VanDerPol FMU from https://github.com/modelica/Reference-FMUs and can trigger an error in FMI.fmiFreeInstance!.

import FMI
import FMIImport

function runAll()
  fmuPath = "fmu/2.0/VanDerPol.fmu"
  inputVars = ["x0", "x1"]
  outputVars = ["x0", "x1"]

  nInputs = length(inputVars)
  nOutputs = length(outputVars)
  nVars = nInputs + nOutputs

  fmu = FMI.fmiLoad(fmuPath)

  Threads.@threads for _ in 1:Threads.nthreads()*10
    @info "Thread $(Threads.threadid()) starting!"
    comp = FMI.fmiInstantiate!(fmu; loggingOn = false, externalCallbacks=false)
    FMI.fmiSetupExperiment(comp)
    FMI.fmiEnterInitializationMode(comp)
    FMI.fmiExitInitializationMode(comp)

    # Set random data
    row = Array{Float64}(undef, nVars)
    row_vr = FMI.fmiStringToValueReference(fmu.modelDescription, vcat(inputVars,outputVars))

    for _ in 1:1000
      row[1:nInputs] = rand(nInputs)

      FMIImport.fmi2SetReal(comp, row_vr, row)
      row[nInputs+1:end] .= FMIImport.fmi2GetReal(comp, row_vr[nInputs+1:end])
    end

    FMI.fmiFreeInstance!(comp)
    @info "Thread $(Threads.threadid()) finished!"
  end

  FMI.fmiUnload(fmu)
end
julia --threads=auto -e "include(\"runAllTheFMUs.jl\"); runAll()"

[ Info: Thread 10 finished!
ERROR: TaskFailedException

    nested task error: AssertionError: fmi2FreeInstance!(...): Freeing 2 instances with one call, this is not allowed. Target address `Ptr{Nothing} @0x00007f2d0401a2a0` was found 2 times at indicies [6, 7].
    Stacktrace:
     [1] (::FMIImport.var"#78#80"{FMICore.FMU2Component{FMICore.FMU2}, Ptr{Nothing}})()
       @ FMIImport ~/.julia/packages/FMIImport/hLBHK/src/FMI2/c.jl:126
     [2] lock(f::FMIImport.var"#78#80"{FMICore.FMU2Component{FMICore.FMU2}, Ptr{Nothing}}, l::ReentrantLock)
       @ Base ./lock.jl:229
     [3] fmi2FreeInstance!(c::FMICore.FMU2Component{FMICore.FMU2}; popComponent::Bool)
       @ FMIImport ~/.julia/packages/FMIImport/hLBHK/src/FMI2/c.jl:124
     [4] fmi2FreeInstance!
       @ ~/.julia/packages/FMIImport/hLBHK/src/FMI2/c.jl:117 [inlined]
     [5] fmiFreeInstance!(str::FMICore.FMU2Component{FMICore.FMU2})
       @ FMI ~/.julia/packages/FMI/1VUBe/src/deprecated.jl:75
     [6] macro expansion
       @ ~/workspace/Testitesttest/FMI-multithread/runAllTheFMUs.jl:39 [inlined]
     [7] (::var"#6#threadsfor_fun#4"{var"#6#threadsfor_fun#3#5"{FMICore.FMU2, Int64, Int64, Vector{String}, Vector{String}, UnitRange{Int64}}})(tid::Int64; onethread::Bool)
       @ Main ./threadingconstructs.jl:163
     [8] #6#threadsfor_fun
       @ ./threadingconstructs.jl:130 [inlined]
     [9] (::Base.Threads.var"#1#2"{var"#6#threadsfor_fun#4"{var"#6#threadsfor_fun#3#5"{FMICore.FMU2, Int64, Int64, Vector{String}, Vector{String}, UnitRange{Int64}}}, Int64})()
       @ Base.Threads ./threadingconstructs.jl:108

...and 1 more exception.

Stacktrace:
 [1] threading_run(fun::var"#6#threadsfor_fun#4"{var"#6#threadsfor_fun#3#5"{FMICore.FMU2, Int64, Int64, Vector{String}, Vector{String}, UnitRange{Int64}}}, static::Bool)
   @ Base.Threads ./threadingconstructs.jl:120
 [2] macro expansion
   @ ./threadingconstructs.jl:168 [inlined]
 [3] runAll()
   @ Main ~/workspace/Testitesttest/FMI-multithread/runAllTheFMUs.jl:21
 [4] top-level scope
   @ none:1

I need to run it multiple times to see the error.

AnHeuermann commented 1 year ago

Maybe it's more of an OpenModelica issue. What are the requirements for the FMU to be simulated in parallel? The OpenModelica FMUs are not thread-safe and I think are using some global variables that shouldn't be changed by other threads. But would this happen in this case?

ThummeTo commented 12 months ago

I think thread-safety is definitely a hard requirement to simulate FMUs multi-threaded. Ignoring this may lead to wrong simulation results at least (but can also crash of coure, dependent on the implementation).

However, the second issue (fmi2FreeInstance!) shouldn't appear and will be investigated.