Convert SBML importer to ElementTree

[MichaelClerx]

minidom is not very good with namespaces

[DavAug]

There seems to be no direct analog of the 'notes' conversion in xlm.elementtree

x = dom_child(xmodel, 'notes')
if x:
      log.log('Converting <model> notes to ascii')
      model.meta['desc'] = html2ascii(x.toxml(), width=75)

I don't have much experience with format conversion tools in python. Do you have a recommendation? I get the notes object with

path = os.path.abspath(os.path.expanduser(path))
tree = ET.parse(path)
root = tree.getroot()
xmodel = root[0]
x = xmodel.find(ns + 'notes')

, where ns is the namespace of the variable.

[MichaelClerx]

Hi @DavAug , not sure if I understand the question correctly, but I think you could convert the contents of the <notes> element to xml again (using https://docs.python.org/2/library/xml.etree.elementtree.html#xml.etree.ElementTree.tostring) and then pass it to html2ascii

[DavAug]

@MichaelClerx tostring(x, method=unicode) function works, however it loses the html formatting when applying html2ascii to it. So the model's desc contains the notes as a huge text block.

I believe that is because tostring assigns the HTML format commands with the namespace 'html'. i.e. <html:body> </html:body>. Would you have a recommendation to turn this off?

[MichaelClerx]

Can you show me an example of the input to html2ascii and the output it produces?

[MichaelClerx]

You can also just leave it as html for now if you like: Just open a new issue for this and we can try to find a solution later

[DavAug]

The notes in HodgkinHuxley.xml are formatted to the following by tostring(x, encoding='undicode', method'xml')

<html:body xmlns:html="http://www.w3.org/1999/xhtml">
        <html:p>This is an implementation of the Hodgkin-Huxley model of the electrical behavior of the squid axon membrane from:      <html:br></html:br>
                <html:b>A quantitative description of membrane current and its application to conduction and excitation in nerve.</html:b>
                <html:br></html:br>
          A. L. Hodgkin and A. F. Huxley. (1952 )      <html:em>Journal of Physiology</html:em>
          119(4): pp 500-544; pmID:      <html:a href="http://www.ncbi.nlm.nih.gov/pubmed/12991237">12991237</html:a>
          .      <html:br></html:br>
                </html:p>
            <html:p>      <html:b>Abstract:</html:b>
                <html:br></html:br>
          This article concludes a series of papers concerned with the flow of electric current through the surface membrane of a giant nerve fibre (Hodgkin,Huxley &amp; Katz, 1952; Hodgkin &amp; Huxley, 1952 a-c). Its general object is to discuss the results of the preceding papers (Part I), to put them into mathematical form (Part II) and to show that they will account for conduction and excitation in quantitative terms (Part III).      </html:p>
            <html:p>This SBML model uses the same formalism as the one described in the paper, contrary to modern versions:      <html:br></html:br>
          * V describes the the membrane depolarisation relative to the resting potential of the membrane      <html:br></html:br>
          * opposing to modern practice, depolarization is      <html:em>negative</html:em>
          , not      <html:em>positive</html:em>
          , so the sign of V is different      <html:br></html:br>
          * inward transmembrane currents are considered positive (inward current positive), contrary to modern use      <html:br></html:br>
          The changeable parameters are the equilibrium potentials(      <html:em>E_R, E_K, E_L, E_Na</html:em>
          ), the membrane depolarization (      <html:em>V</html:em>
          ) and the initial sodium and potassium channel activation and inactivation coefficients (      <html:em>m,h,n</html:em>
          ). The initial values of      <html:em>m,h,n</html:em>
          for the model were calculated for      <html:em>V</html:em>
          = 0 using the equations from the article:      <html:em>n        <html:sub>t=0</html:sub>
            = &#945;_n        <html:sub>V=0</html:sub>
            /(&#945;_n        <html:sub>V=0</html:sub>
            + &#946;_n        <html:sub>V=0</html:sub>
            )        </html:em>
          and equivalent expressions for      <html:em>h</html:em>
          and      <html:em>m</html:em>
          .      <html:br></html:br>
          For single excitations apply a negative membrane depolarization (V &lt; 0). To achieve oscillatory behavior either change the resting potential to a more positive value or apply a constant negative ionic current (I &lt; 0).      <html:br></html:br>
          Two assignments for parameters in the model, alpha_n and alpha_m, are not defined at V=-10 resp. -25 mV. We did not change this to keep the formulas similar to the original publication and as most integrators seem not to have any problem with it. The limits at V=-10 and -25 mV are 0.1 for alpha_n resp. 1 for alpha_m.      <html:br></html:br>
          We thank Mark W. Johnson for finding a bug in the model and his helpful comments.      </html:p>
            <html:p>This model originates from BioModels Database: A Database of Annotated Published Models. It is copyright (c) 2005-2009 The BioModels Team.      <html:br></html:br>
          For more information see the      <html:a href="http://www.ebi.ac.uk/biomodels/legal.html" target="_blank">terms of use</html:a>
          .      <html:br></html:br>
          To cite BioModels Database, please use      <html:a href="http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&amp;pubmedid=16381960" target="_blank">Le Nov&#232;re N., Bornstein B., Broicher A., Courtot M., Donizelli M., Dharuri H., Li L., Sauro H., Schilstra M., Shapiro B., Snoep J.L., Hucka M. (2006) BioModels Database: A Free, Centralized Database of Curated, Published, Quantitative Kinetic Models of Biochemical and Cellular Systems Nucleic Acids Res., 34: D689-D691.</html:a>
                </html:p>
            </html:body>

Converting this with html2ascii gives then the following in the mmt file

desc: """
    This is an implementation of the Hodgkin-Huxley model of the electrical
    behavior of the squid axon membrane from: A quantitative description of
    membrane current and its application to conduction and excitation in nerve.
    A. L. Hodgkin and A. F. Huxley. (1952 ) Journal of Physiology 119(4): pp
    500-544; pmID: 12991237 . Abstract: This article concludes a series of
    papers concerned with the flow of electric current through the surface
    membrane of a giant nerve fibre (Hodgkin,Huxley & Katz, 1952; Hodgkin &
    Huxley, 1952 a-c). Its general object is to discuss the results of the
    preceding papers (Part I), to put them into mathematical form (Part II) and
    to show that they will account for conduction and excitation in
    quantitative terms (Part III). This SBML model uses the same formalism as
    the one described in the paper, contrary to modern versions: * V describes
    the the membrane depolarisation relative to the resting potential of the
    membrane * opposing to modern practice, depolarization is negative , not
    positive , so the sign of V is different * inward transmembrane currents
    are considered positive (inward current positive), contrary to modern use
    The changeable parameters are the equilibrium potentials( E_R, E_K, E_L,
    E_Na ), the membrane depolarization ( V ) and the initial sodium and
    potassium channel activation and inactivation coefficients ( m,h,n ). The
    initial values of m,h,n for the model were calculated for V = 0 using the
    equations from the article: n t=0 = _n V=0 /(_n V=0 + _n V=0 ) and
    equivalent expressions for h and m . For single excitations apply a
    negative membrane depolarization (V < 0). To achieve oscillatory behavior
    either change the resting potential to a more positive value or apply a
    constant negative ionic current (I < 0). Two assignments for parameters in
    the model, alpha_n and alpha_m, are not defined at V=-10 resp. -25 mV. We
    did not change this to keep the formulas similar to the original
    publication and as most integrators seem not to have any problem with it.
    The limits at V=-10 and -25 mV are 0.1 for alpha_n resp. 1 for alpha_m. We
    thank Mark W. Johnson for finding a bug in the model and his helpful
    comments. This model originates from BioModels Database: A Database of
    Annotated Published Models. It is copyright (c) 2005-2009 The BioModels
    Team. For more information see the terms of use . To cite BioModels
    Database, please use Le Novre N., Bornstein B., Broicher A., Courtot M.,
    Donizelli M., Dharuri H., Li L., Sauro H., Schilstra M., Shapiro B., Snoep
    J.L., Hucka M. (2006) BioModels Database: A Free, Centralized Database of
    Curated, Published, Quantitative Kinetic Models of Biochemical and Cellular
    Systems Nucleic Acids Res., 34: D689-D691.
    """

[DavAug]

I'm happy to move this to another issue.

[MichaelClerx]

Thanks!

[DavAug]

@MichaelClerx I finished a version of the SBML importer with etree that reproduces the HodkinHuxley myokit model and passes the test. I will properly test and document the code tomorrow.

I was wondering whether you have suggestions how to best deal with the namespaces in the SMBL files? It seems that we could use predefined namespaces to ensure compatibility, or would that be too restrictive? I would otherwise just hard code namespaces that would be supported, i.e. sbml version, mathml, xhtml etc., only add the entity to the model when it is listed under the namespace and add a log if a namespace occurred that was not supported.

The reason why I would favour predefined namespaces is that etree doesn't seem to make it easy to access the namespaces, other than iterating through the entire tree, which is probably not what we want. And we need the namespace to access the children of a node.

Let me know what you think. :)

[MichaelClerx]

Hi @DavAug !

That's good to hear!

I agree hardcoding namespaces is the way to go. I'd hardcode http://www.sbml.org/sbml/level3/version2/core and raise an error if the anything that isn't in this namespace.

If you wanted to be really strict you could create a method like this one and call it on every element you parse

[DavAug]

Hi @MichaelClerx Happy Easter! I was writing some tests for the new importer and suddenly realised that there are some import errors with some myokit modules, even in the master. I installed the package with pip install -e .[docs,dev,pyqt] and then ran the minimal tests python run_tests.py --minimal. A bunch of seemingly similar errors pop up:

======================================================================
ERROR: test_simulation_p (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_simulation_p
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/david/Workspace/sabs-r3/myokit/myokit/tests/test_simulation_p.py", line 17, in <module>
    from shared import DIR_DATA, CancellingReporter
  File "/Users/david/Workspace/sabs-r3/myokit/myokit/tests/shared.py", line 33, in <module>
    OpenCL_FOUND = myokit.OpenCL.supported()
  File "/Users/david/Workspace/sabs-r3/myokit/myokit/_sim/opencl.py", line 206, in supported
    OpenCL._get_instance()
  File "/Users/david/Workspace/sabs-r3/myokit/myokit/_sim/opencl.py", line 90, in _get_instance
    OpenCL()
  File "/Users/david/Workspace/sabs-r3/myokit/myokit/_sim/opencl.py", line 65, in __init__
    mname, fname, args, libs, libd, incd)
  File "/Users/david/Workspace/sabs-r3/myokit/myokit/_sim/__init__.py", line 197, in _compile
    return load_module(name, d_modul)
  File "/Users/david/Workspace/sabs-r3/myokit/myokit/_sim/__init__.py", line 54, in load_module
    module = importlib.util.module_from_spec(spec)
ImportError: dlopen(/var/folders/8w/b9lb3wb92c1fscyr5tc8nswh0000gp/T/tmp72ws9tb3myokit/module/myokit_opencl_info_36.cpython-37m-darwin.so, 2): Symbol not found: _clGetDeviceIDs
  Referenced from: /var/folders/8w/b9lb3wb92c1fscyr5tc8nswh0000gp/T/tmp72ws9tb3myokit/module/myokit_opencl_info_36.cpython-37m-darwin.so
  Expected in: flat namespace
 in /var/folders/8w/b9lb3wb92c1fscyr5tc8nswh0000gp/T/tmp72ws9tb3myokit/module/myokit_opencl_info_36.cpython-37m-darwin.so

Have you had this problem before?

[MichaelClerx]

That's interesting, looks like OpenCL errors! Usually it wouldn't run those tests unless you'd installed OpenCL drivers specifically. Are you on os/x?

[DavAug]

Yes. I am on Catalina.

[DavAug]

I created a pull request #516

[MichaelClerx]

Fixed in #516.