Reading Allen Cell Types files

[nicain]

Problem/Use Case

Implement backwards-compatible read capability for Allen Cell Types Database files:

from allensdk.core.cell_types_cache import CellTypesCache
ctc = CellTypesCache(manifest_file='boc/manifest.json')
cell_specimen_id = 464212183
data_set = ctc.get_ephys_data(cell_specimen_id)

legacy_map = pynwb.legacy.get_type_map()
io2 = pynwb.NWBHDF5IO(data_set.file_name, extensions=legacy_map, mode='r')
nwb_data = io2.read()

Checklist

• [x] Have you ensured the feature or change was not already reported ?
• [x] Have you included a brief and descriptive title?
• [x] Have you included a clear description of the problem you are trying to solve?
• [x] Have you included a minimal code snippet that reproduces the issue you are encountering?

[nicain]

@ajtritt Ill take a first-pass at this, and let you know where I get stuck

[nicain]

It looks like the biggest stumbling block to reading these files is the use of "Custom" neurodata_type in the files, in unexpected places. For example, in /stimulus/templates, the "gain" dataset in the CurrentClampStimulusSeries has this Custom neurodata_type, triggering the construction of a builder.

[oruebel]

@nicain can this issue be closed?

[nicain]

Unfortunately not :)

[rgerkin]

@nicain This is still a problem right? I tried reading an Allen Cell Types NWB file and it didn't work. Allen Cell Types being the flagship example of an NWB file, I think it would be great if this library supported reading from those files (I'll keep using AllenSDK in the meantime).

FYI, this is related to some "lab" materials I am making for the class I'm teaching, working with NWB files and Cell Types datasets.

[nicain]

CC @sgratiy

[dyf]

@rgerkin Agreed on all counts (would love to see your course materials, btw). We're actually in the process of migrating Cell Types Database files over to NWB2 proper this year.

Time permitting we can look into supporting a pynwb adapter for our various NWB1.x files, although it's a challenging bit of work due to how the schema has evolved over the course of that project. Should be much easier now that the NWB2 schema is stable, though.

[blogeman]

Hi all, checking to see if there is support for NWB1.x files. Specifically I'd like to read the Allen Institute Cell Types Databases as mentioned above. Is there a way to do this?

[bendichter]

@blogeman unfortunately we have not been able to keep up support for 1.x files. I'd suggest using h5py.