search

paulflang / SBML2Julia

A tool to for optimizing parameters of ordinary differential equation (ODE) models. SBML2Julia translates a model from SBML/PEtab format into Julia for Mathematical Programming (JuMP), performs the optimization task and returns the results.
https://sbml2julia.readthedocs.io/en/latest/
MIT License
5 stars 1 forks source link

Failing Benchmark-Models-PEtab cases #8

Open paulflang opened 4 years ago

paulflang commented 4 years ago

This issue regards the petab_suite branch. I tried the Benchmark-Models-PEtab cases but Ipopt ran into problems with a few cases. @sshin23 : could you please download Benchmark-Models-PEtab and have a look what's wrong with this code:

20200810_test_petab_benchmark.txt (also to be found in the tests directory)

You can find more details about which cases cause what problems in line 30. If you get a NotImplementedError please temporarily comment out the lines where it is raised. If the resulting Ipopt failure is not related to the not implemented feature, we should try fixing it. Otherwise we can ignore it.

sshin23 commented 3 years ago

Hey Paul,

I tried python3 test_petab_benchmark.py but got this error message.

Weber_BMC2015
INFO:__main__:Case Weber_BMC2015
Initialising problem...
ERROR:__main__:Case Weber_BMC2015 failed.
ERROR:__main__:[Errno 2] No such file or directory: 'julia'

Am I doing something wrong here?

paulflang commented 3 years ago

I think the error originates in the DisFit Problem constructor (core.DisFitProblem.__init__()) when calling self._jl = Julia(compiled_modules=False). Could you check if you can call Julia from Python like so:

In [1]: from julia.api import Julia                                                                                                                                                                                
In [2]: jl = Julia(compiled_modules=False)
paulflang commented 3 years ago

I would assume that this does not work. Where are you calling Python from? Are you in a virtual Python environment? If so, have you installed Julia to the environment as described in the Dockerfile? If not, can you either give this a try or pull the DisFit Docker image (docker pull paulflang/disfit:latest), create a container (docker run -it paulflang/disfit:latest) and run Python from within the container?

sshin23 commented 3 years ago

Actually, this was a PyJulia issue (I recently reinstalled Julia and PyJulia needed to be configured again). I'm looking into it now.

sshin23 commented 3 years ago

Hey Paul,

This is what I'm getting from DisFit after commenting out the line

        if i != 19: # 6: Ask Sungho why NLP Error again. # 8: What is the memory problem. # 10 Takes to long to be run. Process is killed (at least if run from python)
            # 14: Why is the process killed?, 18: EXIT: Restoration Failed!.
            continue
$ python3 test_petab_benchmark.py
Bachmann_MSB2011
INFO:__main__:Case Bachmann_MSB2011
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Bachmann_MSB2011 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Zheng_PNAS2012
INFO:__main__:Case Zheng_PNAS2012
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Zheng_PNAS2012 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Crauste_CellSystems2017
INFO:__main__:Case Crauste_CellSystems2017
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Crauste_CellSystems2017 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
SalazarCavazos_MBoC2020
INFO:__main__:Case SalazarCavazos_MBoC2020
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case SalazarCavazos_MBoC2020 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Sneyd_PNAS2002
INFO:__main__:Case Sneyd_PNAS2002
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Sneyd_PNAS2002 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
checkBenchmarkModels.py
INFO:__main__:Case checkBenchmarkModels.py
Initialising problem...
ERROR:__main__:Case checkBenchmarkModels.py failed.
ERROR:__main__:[Errno 20] Not a directory: '../Benchmark-Models-PEtab/Benchmark-Models/checkBenchmarkModels.py/checkBenchmarkModels.py.yaml'
Borghans_BiophysChem1997
INFO:__main__:Case Borghans_BiophysChem1997
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Borghans_BiophysChem1997 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Bruno_JExpBio2016
INFO:__main__:Case Bruno_JExpBio2016
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Bruno_JExpBio2016 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Alkan_SciSignal2018
INFO:__main__:Case Alkan_SciSignal2018
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Alkan_SciSignal2018 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Perelson_Science1996
INFO:__main__:Case Perelson_Science1996
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Perelson_Science1996 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Fiedler_BMC2016
INFO:__main__:Case Fiedler_BMC2016
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Fiedler_BMC2016 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Brannmark_JBC2010
INFO:__main__:Case Brannmark_JBC2010
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Brannmark_JBC2010 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Chen_MSB2009
INFO:__main__:Case Chen_MSB2009
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Chen_MSB2009 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Fujita_SciSignal2010
INFO:__main__:Case Fujita_SciSignal2010
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Fujita_SciSignal2010 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Elowitz_Nature2000
INFO:__main__:Case Elowitz_Nature2000
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Elowitz_Nature2000 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Boehm_JProteomeRes2014
INFO:__main__:Case Boehm_JProteomeRes2014
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Boehm_JProteomeRes2014 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Beer_MolBioSystems2014
INFO:__main__:Case Beer_MolBioSystems2014
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Beer_MolBioSystems2014 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Rahman_MBS2016
INFO:__main__:Case Rahman_MBS2016
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Rahman_MBS2016 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Weber_BMC2015
INFO:__main__:Case Weber_BMC2015
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Weber_BMC2015 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Blasi_CellSystems2016
INFO:__main__:Case Blasi_CellSystems2016
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Blasi_CellSystems2016 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).
Schwen_PONE2014
INFO:__main__:Case Schwen_PONE2014
Initialising problem...
INFO:petab.lint:Checking SBML model...
INFO:petab.lint:Checking measurement table...
INFO:petab.lint:Checking condition table...
INFO:petab.lint:Checking observable table...
INFO:petab.lint:Checking parameter table...
INFO:petab.lint:OK
INFO:__main__:Case Schwen_PONE2014 expectedly failed. Required functionality is not implemented: Preequilibration is not implemented (DisFit does not simulate ODEs. Therefore it cannot determine the time until equilibration).

Could you help me figure out what's going on here? Or could you send me the Julia scripts that are problematic?

paulflang commented 3 years ago

The script tests all possible Benchmark models. In core.py I raised the NotImplementedErrors that you see. Could you please comment out the block where theses errors are raised in core.py (In most cases I incorrectly raised NotImplementedErrors. For instance, the error is incorrectly raised when the measurement table contains a column preequilibrationConditionId even though the column is empty, i.e. no preequilibration is needed. This is corrected in a newer version that is not yet fully debugged and hence not pushed)? Once you have done that, please change if i != 19 to if i != 6. This will write a file called julia_code.jl in your current working directory and run the optimization problem it contains. This will fail and I do not know why (perhaps it is due to things not implemented, but I suspect it is sth. else. So far I've only tested ma27 on these problems. Perhaps ma57 does the trick). For debugging purposes, I suggest you copy the content of julia_code.jl in the REPL and tell me what is going wrong/send the corrected file. Once you have figured out the problem with case 6, please move on to cases 8, 10, 14 and 18.