scidash / neuronunit

A package for data-driven validation of neuron and ion channel models using SciUnit
http://neuronunit.scidash.org
38 stars 24 forks source link

get_neab fails because aibs no longer returns all the relevant information. #158

Closed russelljjarvis closed 6 years ago

russelljjarvis commented 6 years ago

Executing the lines:

dataset_id = 354190013  # Internal ID that AIBS uses for a particular Scnn1a-Tg2-Cre
                        # Primary visual area, layer 5 neuron.
observation = aibs.get_observation(dataset_id,'rheobase')

Returns:

Getting Rheobase cached data value for from AIBS dataset 354190013
['value']
130.0 pA

However something with keys: 'std', and 'sem' is expected instead. Not sure if this has something to do with caching.

~/neuronunit/neuronunit/optimization/get_neab.py in <module>()
     52                         # Primary visual area, layer 5 neuron.
     53 observation = aibs.get_observation(dataset_id,'rheobase')
---> 54 tests += [nu_tests.RheobaseTest(observation=observation)]
     55 import pdb; pdb.set_trace()
     56 

~/neuronunit/neuronunit/tests/base.py in __init__(self, observation, name, **params)
     26                  **params):
     27 
---> 28         super(VmTest,self).__init__(observation,name,**params)
     29         cap = []
     30         for cls in self.__class__.__bases__:

/opt/conda/lib/python3.5/site-packages/sciunit/__init__.py in __init__(self, observation, name, **params)
    195 
    196         self.observation = observation
--> 197         self.validate_observation(observation)
    198 
    199         if self.score_type is None or not issubclass(self.score_type, Score):

~/neuronunit/neuronunit/tests/base.py in validate_observation(self, observation, united_keys, nonunited_keys)
     80                 observation['std'] = observation['sem'] * np.sqrt(observation['n'])
     81             else:
---> 82                 raise sciunit.ObservationError(("Observation must have an 'std' key "
     83                                                 "or both 'sem' and 'n' keys."))
     84         return observation

ObservationError: Observation must have an 'std' key or both 'sem' and 'n' keys.
russelljjarvis commented 6 years ago

Attempting to execute:

https://github.com/rgerkin/IzhikevichModel/blob/master/AIBS.ipynb

reproduces this error, the top of AIBS.ipynb can be regarded as a more minimal working example.

Also on this issue, I wonder if the Allen Brain API has changed again. If you open this URL in a browser, and use control-f there are no fields for std, and error, etc.

http://api.brain-map.org/api/v2/data/query.json?q=model::Specimen,rma::criteria

rgerkin commented 6 years ago

Should be fixed in ee3e5add4b895df0bba406d884eafa62b37a276b