scidash / neuronunit

A package for data-driven validation of neuron and ion channel models using SciUnit
http://neuronunit.scidash.org
38 stars 24 forks source link

newer test class for passive elephant tests breaks parallelism in dask, in the case of brian2 backend. #230

Closed russelljjarvis closed 4 years ago

russelljjarvis commented 4 years ago

The new passive test class that replaces generate_prediction with get_result, and generate_features cannot be parallel mapped with dask, as it causes an error: "cannot pickle weak reference".

The passive test class cannot be pickled using the pickle class either, for the same reason, I believe.

russelljjarvis commented 4 years ago

This is only the case when the backend is brian2, and the model is adaptive exponential.

I think the reason is the passive tests try to store the model/simulator as a class attribute, and the brian2 module is assigned as a class attribute, but it is not picklable.

rgerkin commented 4 years ago

Which "passive test class" are you talking about?

Potential solution (though maybe there is something that should be done in the module itself): All SciUnit objects have an attribute unpicklable, which is a list. If you add the right attributes to that list (as strings with the names of the attributes), it will delete them (in a copy of the object) in the __getstate__ method (which pickle uses), which should then make it picklable. Of course, when unpickled, these attributes won't come automatically come back, but if the backend recreates them (as was the case with NEURONBackend, where initializing the simulation in each thread adds the h namespace back), then you should still be able to pickle and unpickle without a problem.

Don't worry about this right now.

russelljjarvis commented 4 years ago

Yes, I found a work around. Brian2 adexp is currently the slowest backend however, so it will hinder long term performance

russelljjarvis commented 4 years ago

This was fixed. I mistakenly stored a brian2 model using self.model inside my divergent version of the passive.py test class file. Code is fast again now including passive tests.