search

scidash / neuronunit

A package for data-driven validation of neuron and ion channel models using SciUnit
http://neuronunit.scidash.org
38 stars 24 forks source link

Create tests corresponding to AIBS and BBP stimulation protocols #71

Open rgerkin opened 7 years ago

rgerkin commented 7 years ago

Single cell somatic current injection protocols for the Allen Institute Cell Types database and for Blue Brain are published and should be used to create analogous tests.

JustasB commented 7 years ago

CC @russelljjarvis @rgerkin

We should create a new subfolder under neuronunit for protocols.

image

The reason is that a given test can be split into two parts: 1) generate prediction and 2) compute score. Prediction generation can be further split into: 1) run protocol and 2) extract features.

In all the tests thus far, the protocol has been "inject square current". Features have been things like tau, rin, vrest, AP threshold, AP amplitude, and AP half-width.

It has become a bit more clear that a separate protocol category is needed with rheobase current test. There the protocol is not just a single square current pulse, but a series of square pulses. And even that can be viewed as two protocols: one single threaded sequential search, and a parallel search.

In the case of my work on channel protocols, and now the allen cell protocols, there are no clear tests or predictions, and only various channel and cell protocols, but I need a place to store them.

I propose to create a sub-folder "protocols" alongside tests, models, and capabilities, which would contain python files for each of the channel and cell protocols. At some point, we should also refactor the existing codebase and move the existing "square pulse" and "series of square pulses to find the rheobase current" protocols in that folder as well. At that point, the folder would contain an expandable library of channel, cell, square pulse, and rheobase search protocols that are usable by existing and future tests.

What are your thoughts? Even if we don't put a sub-folder there, I still need a good place to put the channel and cell protocols, and preferably this should be a place that could capture the existing protocols (square and rheobase search) and be expandable with future protocols.

russelljjarvis commented 7 years ago

I agree with this Justas. Protocals and waveform features are different types of things so they should be separated out.

rgerkin commented 7 years ago

@JustasB I agree. Make it so.

rgerkin commented 7 years ago

@JustasB Have you made it so?

rgerkin commented 6 years ago

@JustasB Status report?

JustasB commented 6 years ago

I worked on the protocols in a separate private repo. My plan was to wait till you guys were finished with most of the large changes and integrate my protocols into NU then. With all the thrashing between my projects, I don't remember where I left off.

Either way, I think this should still be the goal as the fundamental separation between the protocol and feature extraction is still there.

rgerkin commented 5 years ago

@JustasB Do you consider this done for AIBS stimuli? I know you are running them all for NeuroML-DB (not as tests against data per se, but just as a way to run the stimulus and then store the result). Is this integrated into one of the branches of NeuronUnit somewhere?

JustasB commented 5 years ago

@rgerkin

In short: Do you consider this done for AIBS stimuli? Yes, but in the nmldb codebase. Is this integrated into one of the branches of NeuronUnit somewhere? Only the druckmann properties are in NU.

Longer: I think overall the idea that if a set of tests uses a given protocol then that protocol should not be duplicated, but instead separated out, stored under say the "protocols" folder, and reused still makes sense.

On a practical level, when I was working on the AIBS protocols, the NU codebase was in a state of relative flux, and I implemented the protocols in NMLDB repo. When I got to the HBP Druckmann tests, the codebase had stabilized and I implemented the tests in NU. However, the protocols are still implemented in NMLDB.

rgerkin commented 5 years ago

OK, either way I can't close this issue (due to the need for BBP versions), but I think the de-duplication of protocols can also be implemented by sub-classing. For example, if the protocol is basically defined in get_prediction, then there could be protocols classes each with their own get_prediction (or something else that the test's get_prediction calls, and then Tests could subclass from that.