rutishauserlab / recogmem-release-NWB

A NWB-based Dataset and Processing Pipeline of Human Single-Neuron Activity During a Declarative Memory Task
Other
18 stars 11 forks source link

new releases should work #1

Open bendichter opened 4 years ago

bendichter commented 4 years ago

Could you test the code with the latest releases for pynwb and hdmf? I think it should work now

bendichter commented 4 years ago

actually the dev branches should work, but I'm not sure about the current releases. I'll let you know when there's a new release.

nandchandravadia commented 4 years ago

I updated to pynwb == pynwb-1.1.0.post0.dev5 & hdmf == 1.3.2.post0.dev6 I had no problems during the read out of the NWB files; however, during the export (write out), I encountered this error:

Traceback (most recent call last): File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/hdf5/h5tools.py", line 966, in __list_fill__ dset = parent.create_dataset(name, shape=data_shape, dtype=dtype, io_settings) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/h5py/_hl/group.py", line 105, in create_dataset dsid = dataset.make_new_dset(self, shape, dtype, data, kwds) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/h5py/_hl/dataset.py", line 93, in make_new_dset tid = h5t.py_create(dtype, logical=1) File "h5py/h5t.pyx", line 1450, in h5py.h5t.py_create (-------src-dir--------/h5py/h5t.c:16211) File "h5py/h5t.pyx", line 1470, in h5py.h5t.py_create (-------src-dir--------/h5py/h5t.c:16042) File "h5py/h5t.pyx", line 1531, in h5py.h5t.py_create (-------src-dir--------/h5py/h5t.c:16002) TypeError: No conversion path for dtype: dtype('<U')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "", line 1, in File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/RutishauserLabtoNWB/events/newolddelay/python/export/no2nwb_main.py", line 81, in NO2NWB_export io.write(nwbfile, cache_spec = False) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, *parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/hdf5/h5tools.py", line 266, in write call_docval_func(super(HDF5IO, self).write, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(fargs, fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/io.py", line 42, in write self.write_builder(f_builder, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/hdf5/h5tools.py", line 495, in write_builder self.write_group(self.file, gbldr) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/hdf5/h5tools.py", line 643, in write_group self.write_group(group, sub_builder) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/hdf5/h5tools.py", line 648, in write_group self.write_dataset(group, sub_builder) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, **parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/hdf5/h5tools.py", line 842, in write_dataset dset = self.list_fill(parent, name, data, options) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/hdf5/h5tools.py", line 970, in list_fill__ raise_from(Exception(msg), exc) File "", line 2, in raise_from Exception: Could not create dataset categoryname in /intervals/trials with shape (200,), dtype <class 'numpy.str'>, and iosettings {}. No conversion path for dtype: dtype('<U')

nandchandravadia commented 4 years ago

When using the updated packages (those published on PyPi), hdmf == 1.3.2 & pynwb == 1.1.0, I encounter the following error during the export (write out) of the NWB files:

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/RutishauserLabtoNWB/events/newolddelay/python/export/no2nwb_main.py", line 64, in NO2NWB_export nwbfile = no2nwb.no2nwb(NOdata, session_nr, subjects_ini, path_to_data) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/RutishauserLabtoNWB/events/newolddelay/python/export/no2nwb.py", line 241, in no2nwb nwbfile.add_trial_column('stim_on_time', 'the time when the stimulus is shown') File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, **parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/file.py", line 527, in add_trial_column self.__check_trials() File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/file.py", line 519, in __check_trials self.trials = TimeIntervals('trials', 'experimental trials') File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, **parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/epoch.py", line 31, in __init__ call_docval_func(super(TimeIntervals, self).__init__, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(*fargs, **fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, **parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/core.py", line 942, in __init__ id = ElementIdentifiers('id') File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, **parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/core.py", line 401, in __init__ call_docval_func(super(ElementIdentifiers, self).__init__, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(*fargs, **fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, **parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/core.py", line 281, in __init__ call_docval_func(super(NWBData, self).__init__, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(*fargs, **fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, **parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/core.py", line 34, in __init__ call_docval_func(super(NWBBaseType, self).__init__, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(*fargs, **fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 436, in func_call raise_from(ExceptionType(msg), None) File "<string>", line 2, in raise_from TypeError: None is not allowed for 'data' (expected 'ndarray, list, tuple, Dataset, HDMFDataset or data', not None)

And I get this error (during the read out) of the NWB files:

`Traceback (most recent call last): File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1207, in construct obj.init(kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/core.py", line 401, in init call_docval_func(super(ElementIdentifiers, self).init, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(fargs, fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/core.py", line 281, in init call_docval_func(super(NWBData, self).init, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(fargs, fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/pynwb/core.py", line 34, in init call_docval_func(super(NWBBaseType, self).init, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(*fargs, **fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 436, in func_call raise_from(ExceptionType(msg), None) File "", line 2, in raise_from TypeError: None is not allowed for 'data' (expected 'ndarray, list, tuple, Dataset, HDMFDataset or data', not None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "", line 1, in File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/RutishauserLabtoNWB/events/newolddelay/python/analysis/main.py", line 44, in NO2NWB_analysis behavior.plot_behavioral_graphs(session_files) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/RutishauserLabtoNWB/events/newolddelay/python/analysis/behavior.py", line 64, in plot_behavioral_graphs nwbfile = helper.read(str(filename)) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/RutishauserLabtoNWB/events/newolddelay/python/analysis/helper.py", line 30, in read nwbfile = io.read() File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/hdf5/h5tools.py", line 293, in read return call_docval_func(super(HDF5IO, self).read, kwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 327, in call_docval_func return func(*fargs, fkwargs) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/backends/io.py", line 35, in read container = self.manager.construct(f_builder) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, **parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 208, in construct result = self.type_map.construct(builder, self, None) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1712, in construct return attr_map.construct(builder, build_manager, parent) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1175, in construct subspecs = self.get_subspec_values(builder, self.spec, manager) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1117, in get_subspec_values self.get_sub_builders(groups, spec.groups, manager, ret) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1155, in get_sub_builders ret.update(self.get_subspec_values(sub_builder, subspec, manager)) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1117, in get_subspec_values self.get_sub_builders(groups, spec.groups, manager, ret) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1155, in get_sub_builders ret.update(self.get_subspec_values(sub_builder, subspec, manager)) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1117, in get_subspec_values self.get_sub_builders(groups, spec.groups, manager, ret) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1157, in get_sub_builders ret[subspec] = manager.construct(sub_builder) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 204, in construct result = self.__type_map.construct(builder, self, parent) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1712, in construct return attr_map.construct(builder, build_manager, parent) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1175, in construct subspecs = self.get_subspec_values(builder, self.spec, manager) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1118, in get_subspec_values self.get_sub_builders(datasets, spec.datasets, manager, ret) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1157, in get_sub_builders ret[subspec] = manager.construct(sub_builder) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 204, in construct result = self.__type_map.construct(builder, self, parent) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1712, in construct return attr_map.construct(builder, build_manager, parent) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/utils.py", line 438, in func_call return func(self, parsed['args']) File "/Users/nandchandravadia/anaconda/lib/python3.5/site-packages/hdmf/build/map.py", line 1210, in construct raise_from(Exception(msg), ex) File "", line 2, in raise_from Exception: Could not construct ElementIdentifiers object`

bendichter commented 4 years ago

@nandchandravadia thanks I am seeing this too. I'll raise the issue.