Current status (and is there anything we can do to tackle this long standing issue)

[sanjayankur31]

A ticket to collect the current status of a (semi)-automated mod2neuroml pipeline.

How does this work? How much does this work?

Could using nmodl's parser help?

https://bluebrain.github.io/nmodl/html/notebooks/nmodl-python-tutorial.html#Easy-code-generation-using-AST-visitors

[sanjayankur31]

@borismarin : would you know how far we got here please?

[borismarin]

Hey @sanjayankur31

I whipped up a nmodl parser a while back (though it's been gathering some dust lately), that was originally intended to extract info from mod files in a systematic way. In theory, its intermediate representation could be used to compile nmodl to LEMS, but that is a very ambitious goal and there are loads of compromises to be made (eg arbitrary C in VERBATIM blocks, unsupported imperative constructs in Lems).

Now that bluebrain has released their "official" parser, that should be the "correct" way to go if you want to have a structured representation of a mod file.

All that being said, this ticket pertains to mod2neuroml, and not mod2lems conversion. There are a bunch of heuristics (that we employ when manually converting models) that can surely be automated -- but parts of the proccess will never be fully automatic, as there is more information (eg intermediate variables for numerical / programming convenience -- such as the infamous "trap statements" in many rate eq calculations) in mod files than what is needed to produce a 'clean' neuroml description, including "untranslatable" constructs.

Since I've delved into this stuff before, it'd be great to have a longer discussion (and maybe bring Padraig into the mix too) and explore the possibility of reviving this project. I'm pretty confident it would catch the interest of CS-oriented folks looking to contribute to neuroml.

[borismarin]

fyi, check this test to see the kinds of things that we used to be able to do.

[sanjayankur31]

Hi @borismarin , thanks for that explanation. Yeh, conversion of mod files still seems to be the most time consuming bit, so if we can even semi-automate that, it'll mean that a lot more people (that dont know NeuroML/LEMS in detail like us) will be able to then convert models more easily.

I'll put this on our list of TODOs. Padraig returns next week but then I go away for the easter break and we have HARMONY right after. So we'll probably only be able to delve into this after HARMONY.

I was looking into this because we may have a GSoC candidate working on model conversion again. They're a modelling PhD candidate so they know NEURON and all that, but they're new to NeuroML. So, I was looking at ways of making the conversion pipeline a little easier for them.

I'm also working on a NeuroML cell analysis pipeline as a side project, and it'll be good to have some sort of cell conversion pipeline too---perhaps this stuff can even then integrate into NeuroMLDB and all that to allow people to submit cells for conversion/analysis more easily (platforms like ICGenealogy do allow this)

[sanjayankur31]

CC: @pgleeson

I can't seem to put this on our NeuroML backlog. If either of you have the necessary permissions, could you add it there please so it's on our list of things to do?

[pgleeson]

Added our team to this repo...

[borismarin]

@sanjayankur31 I am happy to have a look at my old code and try to update it to use the BlueBrain parser. Feel free to assign me to this task, if it is not too urgent (it might take me a while to find the time).

[sanjayankur31]

Sure, thing. :+1:

[borismarin]

Hey guys, I have been slowly building a proof of concept "compiler" at https://github.com/borismarin/mod2nml . It is still very crude, and will likely fail for any nontrivial modfile. I have been slowly building functionality based on some "classic" mods, see for example this "fully compilable" example which showcases a number of supported constructs (eg detection of std expression forms even if buried into intermediate variables and function calls, detection of fwd/rev kinetics even if they are disguised as tau inf).

That being said, I'm pretty sure that it will break or produce nonsensical results given "regular" mod files found eg on modeldb. The idea here is to see if this kind of approach is feasible, and slowly build functionality based on gradually more complex mod constructs.

[sanjayankur31]

Sounds great. I think if we can come up with something that's semi-automated---it asks users for input when it runs into something it can't handle---that'll be good enough. We're following a similar strategy for the swc to nml converter, where we know that we can't handle/fix all cases that arise in swc files.

[pgleeson]

Looks great! Testing it out in a fork here: https://github.com/pgleeson/mod2nml/actions. Main issue is pip installable version of nmodl is not available on many platforms (incl my macbook). Will look into it further, re building it from source...

[borismarin]

@pgleeson great! but don't get too excited - this is WIP and will probably fail for most nmodl files at this stage - consider it just a proof of concept (even the tests for the currently "supported" mod files still need to be properly written). I will be slowly adding more functionality and polishing things up, but feel free to create issues.

[borismarin]

@pgleeson BTW, there are a number of tests already - try running pytest from /tests (could maybe do that from here?