Review specifications of COPASI's capabilities (biosimulators.json)

[jonrkarr]

Hi @copasi @pmendes. As you probably know, we're containerizing COPASI to make it available through BioSimulations. The metadata about COPASI is captured in biosimulators.json. This includes the modeling frameworks, formats, and algorithms that COPASI supports and the parameters of those algorithms. Basically, this file will control how BioSimulations presents COPASI to end users. Could you review this, or ask the appropriate person to review this? We also welcome any feedback on the structure of the file.

[pmendes]

Hello,

I've had a look and asked others to check too. Here is one correction already:

"license": "MIT" is incorrect. We use the Artistic License version 2.0 http://copasi.org/Download/License/

I have to carefully check everything, as I can already see several problems. For example, in none of our stochastic implementations do we have maximum and minimum step sizes. The RK-45 algorithm is not the classical Runge-Kutta-Fehlberg algorithm; it is rather a hybrid that can have continuous and stochastic variables. I will also have to check the SDE solver, as I am not sure it is the reference you cite (ie we have a better reference, even if that may be correct).

More feedback to come in the next couple days.

Pedro

On 4/20/20 1:46 PM, Jonathan Karr wrote:

Hi @copasi https://github.com/copasi @pmendes https://github.com/pmendes. As you probably know, we're containerizing COPASI to make it available through BioSimulations https://biosimulations.dev. The metadata about COPASI is captured in |properties.json| https://github.com/reproducible-biomedical-modeling/Biosimulations_COPASI/blob/develop/properties.json. This includes the modeling frameworks, formats, and algorithms that COPASI supports and the parameters of those algorithms. Basically, this file will control how BioSimulations presents COPASI to end users. Could you review this, or ask the appropriate person to review this? We also welcome any feedback on the structure of the file.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/reproducible-biomedical-modeling/Biosimulations_COPASI/issues/25, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQPEGX7HIK3TUZK73VDHB3RNSC5RANCNFSM4MMUICGQ.

[jonrkarr]

This is very helpful. Akhil and I started this to get it going, but we realize that we need help with these details as they require a lot of insight into COPASI.

[jonrkarr]

I corrected the COPASI license and annotation of the supported algorithms and parameters. See biosimulators.json.

The same information is visible on the BioSimulators website: https://biosimulators.org/simulators/copasi/. Note,

• The algorithm and parameters names and descriptions are drawn from KISAO.
• The unknown algorithm term (KISAO_0000036) is due to an issue with KISAO which we'll investigate.

@pmendes @shoops @fbergmann @copasi can you review this?

License

I corrected the license to Artistic-2.0.

Annotation of algorithms and algorithm parameters

The initial algorithm/parameter annotation that Akhil started had a variety of issues.

I tried to use the COPASI source code to see what KISAO terms COPASI itself uses for SED-ML import and export. For what I could tell, COPASI largely doesn't import and export simulation algorithms and parameters. COPASI appears to only support three algorithm terms and no parameter terms.

Instead, I used the COPASI UI and the online documentation to determine the time course algorithms and algorithm parameters and annotate them with what I felt are the best KISAO terms. In several cases (algorithms and parameters marked with KISAO term "KISAO_0000036", "KISAO_0000088", "KISAO_0000231, "KISAO_0000242", "KISAO_0000243" and "KISAO_0000483"), I felt that KISAO doesn't have a sufficiently descriptive term. I have requested additional KISAO terms for these algorithms and parameters. I will update the specifications when the terms are approved. See #34.

For the hybrid LSODA algorithm, the online documentation show more user-configurable parameters than the UI (v 4.29). I wasn't sure which is correct. See #40.

I couldn't find a reference for the RI5 stochastic Runge-Kutta method. See #39.

I copied the default parameter values from the COPASI UI.

Todos for BioSimulators Team

• [x] Fix the unknown algorithm term (KISAO_0000036) -- this is because KISAO_0000036 is deprecated; this will be fixed by the replacement with a more specific term.
• [x] Update the specifications of the capabilities of COPASI with newly requested KiSAO terms
• [x] Get new KISAO terms approved
• [x] Update the description of COPASI in the BioSimulators database

Questions for COPASI team

• [x] Are there any issues with the annotation of the algorithms and their parameters (missing/extra parameters, chosen KISAO terms, default values, etc.)?
• [x] Is there a reference for the RI5 stochastic Runge-Kutta method? In my request for a KISAO term, I used this Wikipedia page.
• [x] Are the following user-configurable parameters of the hybrid LSODA algorithm? The online documentation describes these parameters, but the UI doesn't show them.
  • Integrate Reduced Model
  • Relative Tolerance
  • Absolute Tolerance
  • Max Internal Step Size

Discussion with COPASI team about how to map KISAO terms to COPASI methods and options

To use COPASI within our framework, we need to map each KISAO algorithm and algorithm parameter term to the appropriate COPASI method and option. As far as I can tell, COPASI currently has minimal support for KISAO:

• KISAO:0000019 (CVODE):
  • export: all ODE methods
  • import: all Sed time course tasks with algorithm != KISAO:0000241 appear to be mapped to LSODA/LSODAR
• KISAO:0000241 (Gillespie-like method):
  • export: all discrete simulation methods
  • import: all Sed time course tasks with algorithm == KISAO:0000241 appear to be mapped to the Gibson-Bruck Next Reation Method
• KISAO:0000282 (KINSOL) => steady-state
  • export: steady-state methods
  • import: Sed steady-state tasks mapped to a COPASI steady-state task; KISAO term ignored

How would you like to handle this? Implement this mapping into COPASI, or implement this into the BioSimulators layer that Akhil is building on top of COPASI (i.e., this repo)?

[gmarupilla]

FYI, I followed this online documentation.

[fbergmann]

Two answer your questions:

• Issues with KISAO: we only use KISAO to distinguish between, is this a deterministic, stochastic or steady state simulation. In my opinion that is the only thing that is transferable between two implementations. I would hope that simulation results would not for example depend, on one having used a 'forward euler' implementation and it would not be reproducible with a stable solver. Similarly for stochastic methods, without knowing when and how random numbers are drawn (and with what seed) in the individual implementation, getting the same traces again, will be unlikely. So rather than having to implement the ontology lookup (finding out what terms are, and how they are related to something specific we could do), we opted for those terms.
• @shoops should be the right person to produce the reference for the SDE solver.
• setting the tolerance manually, can be done in the ui, as you see in the screenshot here: https://share.getcloudapp.com/xQuAEgpx

As for the question on whether to implement this in COPASI right away, or rather in the your layer on top, i believe for now it would be better to put it in the layer that you are writing. I assume you are talking about:

https://github.com/biosimulators/Biosimulators_COPASI/blob/dev/Biosimulators_copasi/core.py

i could provide you with a function, that you could call after importing the SED-ML, before executing the task, that given an sed-ml simulation object the time course task would be modified with the corresponding parameters. In that i would just encode the specific KISAO terms known to me at the time, once you have a graph of KISAO terms, to find out the closest related term to a given one. If that would be helpful let me know.

Also sorry @gmarupilla, we have not generated the API docs for quite some time and should update them soon.

[jonrkarr]

@fbergmann thanks for the quick reply. Before focusing on whether two simulation tools produce the same results, we first want to make it possible to control each individual simulation tool through SED-ML (specific algorithms and their parameters). We also want to make it clearer which simulation tools support which algorithms and which algorithm parameters are available -- currently this requires digging through a lot of documentation. Once we have this mapped out for multiple simulation tools, we and/or others could use the containers to more easily compare simulation results generated with multiple tools.

i could provide you with a function, that you could call after importing the SED-ML, before executing the task, that given an sed-ml simulation object the time course task would be modified with the corresponding parameters. In that i would just encode the specific KISAO terms known to me at the time, once you have a graph of KISAO terms, to find out the closest related term to a given one. If that would be helpful let me know.

This would be very helpful. tellurium has something like this. We've implemented something similar to this for BioNetGen and GillesPy2. My recommendations for the mapping between KISAO terms and COPASI algorithms/parameters are in https://github.com/biosimulators/Biosimulators_COPASI/blob/dev/biosimulators.json. Note, a few of these need to be swapped with more specific KISAO terms when they are approved. See #38. At this point, I'd like to get any feedback on the mapping.

• Algorithm terms:
  • COPASI name: specs['algorithms'][0]['name']
  • KISAO id: specs['algorithms'][0]['kisaoId']['id']
• Parameter terms:
  • COPASI name: specs['algorithms'][0]['parameters']['name']
  • KISAO id: specs['algorithms'][0]['parameters'][0]['kisaoId']['id']

[shoops]

Hello All,

Please find attached the slides of my talk at COMBINE about SDE. You should find all required information about the algorithm in it.

Thanks, Stefan

On Wed, 2020-10-28 at 09:28 -0700, Frank Bergmann wrote:

Two answer your questions:

Issues with KISAO: we only use KISAO to distinguish between, is this a deterministic, stochastic or steady state simulation. In my opinion that is the only thing that is transferable between two implementations. I would hope that simulation results would not for example depend, on one having used a 'forward euler' implementation and it would not be reproducible with a stable solver. Similarly for stochastic methods, without knowing when and how random numbers are drawn (and with what seed) in the individual implementation, getting the same traces again, will be unlikely. So rather than having to implement the ontology lookup (finding out what terms are, and how they are related to something specific we could do), we opted for those terms. @shoops should be the right person to produce the reference for the SDE solver. setting the tolerance manually, can be done in the ui, as you see in the screenshot here: https://share.getcloudapp.com/xQuAEgpx As for the question on whether to implement this in COPASI right away, or rather in the your layer on top, i believe for now it would be better to put it in the layer that you are writing. I assume you are talking about:

https://github.com/biosimulators/Biosimulators_COPASI/blob/dev/Biosimulators_copasi/core.py

i could provide you with a function, that you could call after importing the SED-ML, before executing the task, that given an sed-ml simulation object the time course task would be modified with the corresponding parameters. In that i would just encode the specific KISAO terms known to me at the time, once you have a graph of KISAO terms, to find out the closest related term to a given one. If that would be helpful let me know.

Also sorry @gmarupilla, we have not generated the API docs for quite some time and should update them soon.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[jonrkarr]

@shoops The attachment seems to be scrubbed by GitHub. Are there any references you'd like to cite for the SDE method?

[shoops]

Hello Jonathan,

Thanks for the reminder why I am not using github issues :) Here is a link to the slides:

https://drive.google.com/file/d/1ByepKNHAMYj4e-h3SjEx3fGe4IiJSJev/view?usp=sharing

Thanks, Stefan

On Wed, 2020-10-28 at 10:52 -0700, Jonathan Karr wrote:

@shoops The attachment seems to be scrubbed by GitHub. Are there any references you'd like to cite for the SDE method?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[jonrkarr]

@shoops I added two references for the SDE method:

• Andreas Rößler. Second order Runge-Kutta methods for Itô stochastic differential equations. SIAM Journal Numerical Analysis 47 (3), 1713–1738 (2009).
• Daniel T. Gillespie. The chemical Langevin equation. Journal Chemical Physics 113, 297 (2000).

I also requested a new KISAO term for RI5.

[fbergmann]

@jonrkarr i have implemented setting method parameters as discussed before, i could make a pull request if you wanted to:

https://github.com/fbergmann/Biosimulations_COPASI/commit/868688bdfb2782df2a9f8718ac3de0b91c826a9e

[jonrkarr]

@fbergmann thanks for outlining the simulation methods and parameters. I filled in the new KiSAO ids and used this to re-write the module.

To confirm, the subtype argument of Gibson-Bruck should be ignored? I ask because the online documentation suggests that the parameter isn't used. However, the parameter is commented out in your pull request.

What's the syntax for setting the 'Partitioning Strategy' and 'Deterministic Reactions' of the Hybrid RK45 method?

[fbergmann]

indeed, the subtype parameter is not used in the code right now. I would suppose it is there for round-tripping models with perhaps Gepasi, but i'm guessing.

The Partitioning Strategy for the hybrid RK45, will be either All Reactions Deterministic (algorithm is running deterministically), All Reactions Stochastic (algorithm is running all reactions stochastically) or User specified Partition (algorithm uses a fixed partition defined below with the selected reactions deterministic, and all others stochastic). For the user specified partition the parameter group Deterministic Reactions contains the names of the reactions to be simulated deterministically.

Or were you asking how to change the partitioning in the python code?

[jonrkarr]

Thanks for clarifying these issues.

I still have one question, what is the format of the parameter group Deterministic Reactions? Does this need to be a list of strings, a string which contains a comma-separated list of ids?

[fbergmann]

For us in COPASI it would be a parameter group (a list of elements), uniquely identifying the reactions:

  <Method name="Hybrid (RK-45)" type="Hybrid (DSA-ODE45)">
    <Parameter name="Max Internal Steps" type="unsignedInteger" value="100000"/>
    <Parameter name="Relative Tolerance" type="unsignedFloat" value="9.9999999999999995e-07"/>
    <Parameter name="Absolute Tolerance" type="unsignedFloat" value="1.0000000000000001e-09"/>
    <Parameter name="Partitioning Strategy" type="string" value="User specified Partition"/>
    <ParameterGroup name="Deterministic Reactions">
      <Parameter name="Reaction" type="cn" value="CN=Root,Model=New Model,Vector=Reactions[reaction]"/>
      <Parameter name="Reaction" type="cn" value="CN=Root,Model=New Model,Vector=Reactions[reaction_2]"/>
    </ParameterGroup>
    <Parameter name="Use Random Seed" type="bool" value="0"/>
    <Parameter name="Random Seed" type="unsignedInteger" value="1"/>
  </Method>

For SBML a comma separated list of reaction ids would make sense. if you wanted to add it to KISAO it would probably have to be a list of strings again (as generic identifiers could contain commas).

[jonrkarr]

We're using the COPASI Python API to set parameter values. Could you give me an example of the code to set the list of deterministic reactions.

Yes, the value would be encoded into KISAO as an escaped comma-separated list of strings (e.g., JSON).

I have already added a KISAO term for a parameter which is a list of ids of reactions. Other tools such as COBRApy and CBMPy also have similar parameters for algorithms such as FVA.

[fbergmann]

In theory the following bit of code (using the conventions from your biosimulations_copasi would set the values:

def _get_reaction_cn_for_sbml_id(model, sbml_id):
    # type: (copasi.CModel, copasi.CCommonName) -> copasi.CRegisteredCommonName
    for reaction in model.getReactions():
        assert (isinstance(reaction, copasi.CReaction))
        if reaction.getSBMLId() == sbml_id:
            return copasi.CRegisteredCommonName(reaction.getCN())

    return None

def _set_deterministic_reactions(method, id_list):
    # type: (copasi.CCopasiMethod, [str]) -> None
    group = method.getParameter('Deterministic Reactions')
    if not group:
        logging.warning('the method does not have the deterministic reactions parameter')
        return

    assert (isinstance(group, copasi.CCopasiParameterGroup))
    datamodel = method.getObjectDataModel()
    method.getParameter('Partitioning Strategy').setStringValue('User specified Partition')  # select custom partitioning
    group.clear()  # remove existing items
    for sbml_id in id_list:
        cn = _get_reaction_cn_for_sbml_id(datamodel.getModel(), sbml_id)
        if cn is None:
            logging.warning('no reaction for sbml_id {0} ... skipping'.format(sbml_id))
            continue
        param = copasi.CCopasiParameter('Reaction', copasi.CCopasiParameter.Type_CN)
        if not param.setCNValue(cn):
            logging.warning("couldn't set reaction cn, a newer version of python-copasi is needed")
        group.addParameter(param)

this could be called if the method was of the correct method type you could call this function with the copasi method, and a list of sbml reaction ids. Unfortunately it is currently not working on my machine, I'll look into that and will likely have to release a new python-copasi version for that.

[jonrkarr]

I'm closing this issue because all questions/development has been done except the issue with the hybrid RK45 method that Frank noted. I'm moving this issue to #43.