search

biosimulators / Biosimulators_COPASI

COPASI biochemical network simulation program via BioSimulators-compliant command-line interface and Docker container
https://docs.biosimulators.org/Biosimulators_COPASI
MIT License
2 stars 3 forks source link

All biomodels Copasi currently fails for #74

Open luciansmith opened 1 year ago

luciansmith commented 1 year ago

tellurium_failures.xlsx copasi_failures.csv

I'm attaching two lists: one of all models where Tellurium currently fails, and one of all models where Copasi currently fails. In general, it should in theory be possible for Copasi to succeed at everything that Tellurium succeeds on, and it might succeed at a few more, but I wouldn't count on it.

The list has links to the logs of all 8/25/23 runs of all biomodels on Copasi, which should tell you what the problems were. Many of them are the same problem; i.e. for runs like

https://run.biosimulations.org/runs/64e932fc548ceb84f3effcf6#tab=log

where the problem is that the number of points, originally 1000, got read into a real value instead of an int, so couldn't be used later. Once that's fixed, many of the other runs will also be fixed. Similar things will be true of the other runs; I'm happy to explain how they work in Tellurium if you like.

luciansmith commented 2 months ago

Update! Here's the comparison for all models, run with version 4.36 (pre-basico) and 4.42 (the latest) copasi_compare.csv

pmendes commented 2 months ago

Note that the models showing failed in version 4.36 are also not due to COPASI itself, but the adapter. For example BIOMD0000000451 : this is a model that was actually written and run in COPASI in the first place. Both the SED-ML and SBML work fine in COPASI 4.36. Obviously it is the adapter and/or the workflow that is causing the failure.

jcschaff commented 2 months ago

note: Lucians runs from 3 days ago (see previous post) compare containers built from biosimulators_copasi with tags 0.1.35 and 0.1.36

luciansmith commented 2 months ago

@pmendes I've assumed that all the failures are from the wrapper. It's possible there might be a corner case here or there due to COPASI itself, but even that is unlikely.

pmendes commented 2 months ago

@luciansmith yes, apart from the models with DDEs, I think we should be passing all of them (perhaps one or other SBML L3V2 that uses one of the new math functions may also fail but I doubt there are many yet)

luciansmith commented 2 months ago

From @CodeByDrescher :

Changes include:

Results attached! copasi_compare.csv

luciansmith commented 2 months ago

Latest fixes mean we're up to 838! ('44b' in the new comparison). copasi_compare.csv

pmendes commented 2 months ago

It's getting better, many of the failures are expected (DDEs), but there are still some that shouldn't be. For example BIOMD0000000676 and BIOMD0000000696 should work (biomodels db used COPASI in curation).

luciansmith commented 2 months ago

Yes, work is ongoing.

luciansmith commented 1 month ago

Another 50 successes! We're up to 888.

copasi_compare.csv

luciansmith commented 1 month ago

And with Frank's fixes to Copasi itself, we got another 100 models, up to 988! copasi_compare.csv

pmendes commented 1 month ago

Great progress!

Just out of curiosity I looked up randomly one that still fails, and I find BIOMD0000000498 which is a model from my own lab (and obviously developed in COPASI itself). Curious to see why this one still fails...

CodeByDrescher commented 1 month ago

BIOMD0000000498 is failed by Biosimulators Utils because the SedML requests `steady state root-finding method' (KISAO_0000407), which none of COPASI's algorithms supposedly fall under. This warrants deeper investigation.

fbergmann commented 1 month ago

This term was introduced long ago, to denote just a steady state algorithm.

https://www.ebi.ac.uk/ols4/ontologies/kisao/classes/http%253A%252F%252Fwww.biomodels.net%252Fkisao%252FKISAO%2523KISAO_0000407

And we definitely have one of those in COPASI. When we import it we just take that to mean to run the steady state task.

luciansmith commented 1 month ago

I had to extensively update KiSAO a little while back to correctly organize the steady state and flux balance algorithms; there was zero support for steady state algorithmic substitution in the library like there was for simulations. So I believe the only thing that now needs to happen is for the copasi wrapper to make use of those new functions from the kisao library.

Also, OLS4 seems to have an out-of-date version of KiSAO, with the older scrambled steady state relationships. I'm not sure how to get those updated; I had thought at one point that pushing things to github also triggered pushing the ontology to various ontology front ends, but maybe OLS4 didn't make that list, or something else broke somehow.

fbergmann commented 1 month ago

@CodeByDrescher even if the definition changed, since COPASI does have a method to solve Nv = 0, which would fall under the definiton f(x) = 0 as stated in the new definition of 407, that qualifies to still be used.

If substitutions are allowed, we should also take all the other ones that tellurium accepts for steadystate, which are: 

[99, 274, 282, 283, 355, 356, 407, 408, 409, 410, 411, 412, 413, 432, 437, 568, 569, ]

In my opinion, as soon as someone demonstrates: "This model cannot be solved in RoadRunner, because it does not implement LSODA" KISAO has failed. And for steady state it should also not matter if someone implements kinsol, nleq or else.

luciansmith commented 1 month ago

Yes, this was the functionality that I added. There were functions in libkisao that let you discover all the various ODE solvers, but there was nothing that let you discover the steady state functions. Now there are! They are now grouped together in the ontology, and there is also a new 'steady state root-finding problem' to parallel the ODE 'ordinary differential equation problem'.

But these didn't exist back when the copasi wrapper was first written, and can now be added (similarly for the other wrappers, too).