Closed russelljjarvis closed 7 years ago
@russelljjarvis
The relevant code is here where the options
dictionary is calculated. Note that the options
dictionary replaces "Backend" with "", so the name stored in each key would be e.g. "jNeuroML" instead of "jNeuroMLBackend". Similarly, the name you want to use for DiskBackend is "Disk" and for MemoryBackend it is "Memory".
Also, local_run()
in your backends appears to make calls that are also made in the real NEURONBackend (in the case where the cached value is not found). Can you change local_run to make a call to super() in that case, so that you don't have duplicated code?
Thanks for setting me straight about the Disk, and Memory way of instancing. Ie the actual Backend part of the string is stripped away.
AOK. I think I see what your saying. I can de-duplicate a lot of code if inside the over-riddenlocal_run
method I call the super(MemoryBackend,self).local_run()
.
It confuses me a bit, because I think of overriding as destructively replacing the parent classes method, but I can sort of see how you could still refer to the parents method anyway.
def local_run(self):
if self.current not in self.model.cached_params:
results = super(MemoryBackend,self).local_run()#
self.model.cached_params[str(current)] = {}
self.model.cached_params[str(current)]=results
return self.model.cached_params[str(current)].results
else:
return self.model.cached_params[str(current)].results
Also on the issue of caching per model attribute, results/V_{s}. I thought, that I should try to verify my claim, that the same model attributes are not re-evaluated frequently enough, that the cost of storing the simulation results would out weigh performance costs of de duplication of workload.
To verify. I could have a dictionary self.model.cached_attrs
that I use to count the number of times the a particular parameter set is evaluated.
With self.model.cached_attrs[str(model.attrs)]+=1
@russelljjarvis I suspect the number will be low but you can certainly check.
I am also addressing this in unit testing: https://github.com/russelljjarvis/neuronunit/blob/dev/unit_test/testNEURONparallel.py#L91-L111
set is an iterable, except unlike list its composed of only unique elements, so if I cast list to set and then get the length, the length of set should not be reduced relative to before the cast if parameter values are really unique. I am performing this operation on a list of 10,000 randomly generated parameter sets, and it is passing.
Its probably more meaningful to cast the lists: pop, and ga_history_pop to set,
unique_check = pop.extend(ga_history_pop)
bfl = len(unique_check)
afl = len(set(unique_check))
self.assertEqual(bfl,afl)
to check that even as the GA population converges on a solution, that the models it's testing are unique (not redundant) enough.
In current use-case of neuronunit, 8 tests all draw from only 2 types of current injection type (2 simulator runs).
Action of the NSGA algorithm is supposed to avoid sampling twice from exactly the same parameter space, so I do not think it is worth pursuing a memory intensive option of storing all V_{m}'s in a dictionary where keys to those dictionaries are model attributes. As alternative to model_attribute:model_results hashing I just want to define a model backend, that stores runs in a dictionary, where the only two keys in the dictionary are two types of current injection used.
To achieve the end of caching runs per current injection type. I created a Memory backend that inherits from
NEURONBackend
, and over writes itsinject_sqaure_current
, andlocal_run
methods. See below:Note also I think this is also the right approach to caching per model attribute simulation runs (something I have no interest in, since I think it will slow down the GA action rather than speeding it up). Any attribute of the neuronunit model, such as a caching dictionary can be shifted into a data transport container (
dtc
) object. A pattern for achieving this would be:dtc.cached_attrs.update(model.cached_attrs)
There are many oppurtunities when all the dtc's exist togethor in a list on CPU0 that list is called
dtcpop
. To federate all of these dictionaries, such that each dtc, knows what its siblings have also previously run, you can just do something like:My feeling is that somewhere in neuronunit there is a list of all the allowed backends, and the only two existing items are jNeuroMLBackend and NEURONbackend.
I wonder if you know of any places I should edit to help neuronunit sense/permit these DiskBackend and MemoryBackend?
The first problem with this approach is getting neuronunit to sense that this backend is available.
Printing dir on the backends module shows all the relevant classes (ie memory backend) etc:
If I try to instantiate with DiskBackend:
If I try to instance with MemoryBackend: