NeurodataWithoutBorders / pynwb

A Python API for working with Neurodata stored in the NWB Format
https://pynwb.readthedocs.io
Other
178 stars 84 forks source link

NWB files with cached extensions cannot be opened in matnwb #913

Closed bendichter closed 5 years ago

bendichter commented 5 years ago

Bug

When a file is generated in pynwb with a cached spec, e.g

from pynwb import NWBHDF5IO, NWBFile
from datetime import datetime

nwbfile = NWBFile('aa', 'aa', datetime.now().astimezone())

with NWBHDF5IO('test_ab.nwb', 'w') as io:
    io.write(nwbfile, cache_spec=True)

it cannot be read in matnwb:

nwbRead('/Users/bendichter/dev/hdmf/test_ab.nwb')
Error using types.core.NWBContainer (line 23)
Unmatched parameter name '.specloc' must be a string
scalar or character vector that can represent a field
name.

Error in types.core.NWBFile (line 105)
        obj =
        obj@types.core.NWBContainer(varargin{:});

Error in nwbfile (line 15)
            obj = obj@types.core.NWBFile(varargin{:});

Error in io.parseGroup (line 64)
        parsed = nwbfile(kwargs{:});

Error in nwbRead (line 20)
    nwb = io.parseGroup(filename, info);

There appears to be a mismatch between the schema and what is actually stored in the HDF5 file. I'm not really sure what the best solution is here, but we need to fix this so that nwb files with cached specs are interoperable between python and matlab. Would it be possible to either document the use of ".specloc" as an optional attribute of NWBFile or to change the way extensions are cached? What is the purpose of ".specloc"?

issue in matnwb repo: https://github.com/NeurodataWithoutBorders/matnwb/issues/42

Checklist

bendichter commented 5 years ago

Cached extensions are currently stored in JSON. What is the reason for this? Would it be possible to store them in YAML instead? As far as I understand this would make things much easier for @ln-vidrio

t-b commented 5 years ago

@bendichter DIscussed yesterday already ;)

Reasons for JSON: