search

AMICI-dev / AMICI

Advanced Multilanguage Interface to CVODES and IDAS
https://amici.readthedocs.io/
Other
107 stars 30 forks source link

Uncaught C exceptions #2478

Closed FFroehlich closed 30 minutes ago

FFroehlich commented 6 days ago 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.991 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.991 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.991 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.991 - amici.swig_wrappers - DEBUG - [cEFM192A__EGF+cEFM192A][CVODES:CVode:OTHER] AMICI ERROR: in module CVODES in function CVode : At t = 0 repeated recoverable sensitivity right-hand side function errors. 
2024-07-05 03:23:14.991 - amici.swig_wrappers - ERROR - [cEFM192A__EGF+cEFM192A][OTHER] AMICI simulation failed: CVODE routine reInitPostProcess failed with error code -43. tout: 5.5, t: 0.
2024-07-05 03:23:15.032 - amici.swig_wrappers - DEBUG - [cEFM192A__EGF+cEFM192A][CVODES:CVode:OTHER] AMICI ERROR: in module CVODES in function CVode : At t = 0, the setup routine failed in an unrecoverable manner. 
2024-07-05 03:23:15.033 - amici.swig_wrappers - ERROR - [cEFM192A__EGF+cEFM192A][OTHER] AMICI simulation failed: CVODE routine reInitPostProcess failed with error code -6. tout: 5.5, t: 0.
terminate called after throwing an instance of 'amici::CvodeException'
  what():  CVODE routine CVodeGetSens failed with error code -25.

While the underlying issue is Inf in some of the equations, this shouldn't lead to an uncaught c++ exception that terminates python.

dweindl commented 3 days ago

Do you have more context where that happened? runAmiciSimulation / runAmiciSimulations should catch everything. Might be again some linking issue such that exceptions in the base extension and model extension don't match?

FFroehlich commented 3 days ago

this was in the context of runAmiciSimulations, but we were unable to reproduce the issues locally. In principle the models should be automatically compiled, we will investigate this further.

dilpath commented 1 day ago

I can reproducibly produce the error below. The error is not unexpected since I am setting up a custom simulator that builds on simulate_petab, and this custom simulator isn't really working yet, but it looks like an uncaught exception too. Let me know if you'd like an MWE.

terminate called after throwing an instance of 'amici::AmiException'
  what():  Number of initial conditions sensitivities (40) in model does not match ExpData (35).
Aborted (core dumped)
dweindl commented 1 day ago 
terminate called after throwing an instance of 'amici::AmiException'
  what():  Number of initial conditions sensitivities (40) in model does not match ExpData (35).
Aborted (core dumped)

Thanks. This shouldn't happen. There should be a Python RuntimeError instead. On my computer, this works as excepted. I added a test case in #2484, does this pass for you?

Potential issues:

dilpath commented 1 day ago

I added a test case in https://github.com/AMICI-dev/AMICI/pull/2484, does this pass for you?

The first version of this PR passed for me. The current version fails with

FAILED test_swig_interface.py::test_python_exceptions - RuntimeError: Dimension mismatch. Size of parameters does not match number of model parameters.

but I guess this still counts as a pass, since there is no uncaught exception.

Potential issues:

* different swig versions

* updating amici without re-installation

* re-installing amici without re-importing the model(?)

* linking issues

The 2nd and 3rd aren't the case for me I think. My SWIG is 4.0.2. Not sure how to check the linking issue. I'll try to create an MWE then.

dweindl commented 1 day ago

The first version of this PR passed for me. The current version fails with

Good. Sorry, fixed the other issue. RuntimeError is good though.

I'll try to create an MWE then.

Yes, that would be interesting then.

dweindl commented 20 hours ago

I'll try to create an MWE then.

Thanks. So in @dilpath's case, the problem is that the error occurs already here, outside try:

https://github.com/AMICI-dev/AMICI/blob/b958cce6a08683104b570b6aaa672d83c2c3fed6/src/amici.cpp#L94

When using runAmiciSimulation directly, this exception will result in a Python RuntimeError which is fine. However, for runAmiciSimulations (used by simulate_petab), this exception must be caught within the simulation thread, otherwise the program will terminate as observed here.

The same issue is lurking here: https://github.com/AMICI-dev/AMICI/blob/b958cce6a08683104b570b6aaa672d83c2c3fed6/src/amici.cpp#L291

The original issue is probably slightly different, but I assume it's also related to using runAmiciSimulations with some exception in runAmiciSimulation not being caught.