search

openworm / CElegansNeuroML

NeuroML based C elegans model, contained in a neuroConstruct project, as well as c302
http://opensourcebrain.org/projects/celegans
134 stars 51 forks source link

Use OpenWormReader to generate connectome #49

Closed travs closed 8 years ago

travs commented 9 years ago

This is closed when we are using PyOpenWorm's data via OpenWormReader (#46) to get the connectome we use in c302.

@ahrasheed

pgleeson commented 9 years ago

This really depends on https://github.com/openworm/PyOpenWorm/issues/177 getting completed.

What's still lacking in the use of these multiple connectome data sources is a rigorous analysis, comparison and documentation of the different data sets, which will become more important as the new connectome is used. Unless there's a well informed paper trail showing that the project knows about the history of data produced in this area (and why we have chosen to use what we do), it will be difficult to convince c elegans physiologists we know what we're talking about...

Maybe there's a need for a separate https://github.com/openworm/ConnectomeDatasets project to document and make scripts to analyse/compare the different data sets out there and can be used as a basis/reference for what's used in PyOpenWorm?

pgleeson commented 8 years ago

Time to revisit this since there has been progress with https://github.com/openworm/PyOpenWorm/issues/177.

Would it make sense to do this already though, before the latest changes in PyOpenWorm are merged to master? Anyone installing from master would get a different connectome than those installing from dev (or pip it seems)...

slarson commented 8 years ago

OK; so now we've got a release to master :)

So folks who have been waiting on this one maybe now we can get back to it. @ahrasheed are you up for it? I will also comment over on #27 which has more on this.

pgleeson commented 8 years ago

OpenWormReader.py has been now updated and is being successfully tested with the latest PyOpenWorm from master.

However, the OpenWormReader takes about 3 minutes to extract the connectivity data using PyOpenWorm. Below is the log from running on Travis-CI

$ python OpenWormReader.py
2016-01-29 14:54:45,436: OpenWormReader - INFO: Initialising OpenWormReader
2016-01-29 14:54:45,469: root - INFO: Connected to database
2016-01-29 14:54:55,024: OpenWormReader - INFO: Finished initialising OpenWormReader
2016-01-29 14:58:15,009: OpenWormReader - INFO: Total cells read 451
2016-01-29 14:58:15,009: OpenWormReader - INFO: Total connections read 7319

This needs to be sped up significantly if it's to be used for generating connections elsewhere. Also, there should be 2 methods (as in SpreadsheetDataReader.py) one for neurons->neurons and one for motorneurons->muscles

slarson commented 8 years ago

Awesome! Should this issue be closed then?

The performance issue is regrettable and has been known for a while: https://github.com/openworm/PyOpenWorm/issues/90

It would be good to turn attention back to it now.

pgleeson commented 8 years ago

Technically this is done now, the next step of using PyOpenWorm/OpenWormReader in c302 is really covered here: https://github.com/openworm/CElegansNeuroML/issues/27. but that is unlikey to happen until performance is improved in some way...