search

esi-neuroscience / oephys2nwb

Export Open Ephys binary data to NWB 2.0
BSD 3-Clause "New" or "Revised" License
3 stars 2 forks source link

Weird namespace error if ndx_events not imported before loading nwb #2

Closed KatharineShapcott closed 1 year ago

KatharineShapcott commented 2 years ago

Here I didn't import ndx_events first, if I do everything is fine.

>>> from pynwb import NWBHDF5IO
>>> with NWBHDF5IO('/gs/home/shapcottk/ZeroNoise/testname.nwb', 'r') as io:
...     nwbfile = io.read()
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/backends/hdf5/h5tools.py", line 498, in read
    return call_docval_func(super().read, kwargs)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/utils.py", line 424, in call_docval_func
    return func(*fargs, **fkwargs)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/utils.py", line 583, in func_call
    return func(args[0], **pargs)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/backends/io.py", line 41, in read
    container = self.__manager.construct(f_builder)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/utils.py", line 583, in func_call
    return func(args[0], **pargs)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/build/manager.py", line 280, in construct
    result = self.__type_map.construct(builder, self, None)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/utils.py", line 583, in func_call
    return func(args[0], **pargs)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/build/manager.py", line 790, in construct
    return obj_mapper.construct(builder, build_manager, parent)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/utils.py", line 583, in func_call
    return func(args[0], **pargs)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/build/objectmapper.py", line 1214, in construct
    subspecs = self.__get_subspec_values(builder, self.spec, manager)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/build/objectmapper.py", line 1143, in __get_subspec_values
    self.__get_sub_builders(groups, spec.groups, manager, ret)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/build/objectmapper.py", line 1194, in __get_sub_builders
    ret.update(self.__get_subspec_values(sub_builder, subspec, manager))
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/build/objectmapper.py", line 1143, in __get_subspec_values
    self.__get_sub_builders(groups, spec.groups, manager, ret)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/build/objectmapper.py", line 1172, in __get_sub_builders
    for parent_dt in manager.namespace_catalog.get_hierarchy(ns, dt):
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/utils.py", line 583, in func_call
    return func(args[0], **pargs)
  File "/gs/home/shapcottk/.conda/envs/openephys/lib/python3.10/site-packages/hdmf/spec/namespace.py", line 329, in get_hierarchy
    raise KeyError("'%s' not a namespace" % namespace)
KeyError: "'ndx-events' not a namespace"

Maybe related to this issue? https://github.com/rly/ndx-events/issues/3

Although if I use load_namespaces=True everything is then fine (unlike in the issue):

>>> with NWBHDF5IO('/gs/home/shapcottk/ZeroNoise/testname.nwb', 'r',load_namespaces=True) as io:
...     nwbfile = io.read()
...     print(nwbfile.acquisition['TTL_Pulses'])
... 
TTL_Pulses abc.TTLs at 0x140301537757072
Fields:
  data: <HDF5 dataset "data": shape (19259,), type "<u2">
  data__labels: ['']
  description: TTL pulse values
  timestamps: <HDF5 dataset "timestamps": shape (19259,), type "<f4">
  timestamps__resolution: 3.3333333333333335e-05
  timestamps__unit: seconds
pantaray commented 2 years ago

Ha, good catch! Yes, seems like rly/ndx-events#3 is the culprit here. Since we cannot really control how people access the data once it's inside the NWB file, do you think it makes sense to mention this in the project README? Given the numerous dependencies the code requires already, I think it might be beneficial to package the script, so people can simply pip install it. What do you think?

KatharineShapcott commented 2 years ago

Yes I think that would be a helpful solution. Especially since open_ephys.analysis isn't on pip yet and has dependencies of its own.