ValueError: No data_type found for builder root

[wuffi]

Trying to convert a directory with data in Neuroscope (Neurosuite) format gives me the error ValueError: No data_type found for builder root in the get_cls method in line 701 of hdmf/build/manager.py.

I had written a minimal custom NWBConverter and an example to reproduce the the error would be:

from buzsaki_lab_to_nwb import MyNWBConverter
import spikeextractors

input_args = {
    'NeuroscopeSorting': {'folder_path': "D:\sample_data\sorted"}
}

my_converter = MyNWBConverter(**input_args)
metadata = my_converter.get_metadata()
nwbfile_path="D:\sample_data\sample_data.nwb"

temp = my_converter.data_interface_objects['NeuroscopeSorting'].sorting_extractor
temp.get_unit_ids()
temp.get_units_spike_train()
spikeextractors.NwbSortingExtractor.write_sorting(temp, save_path=nwbfile_path)

Using the spikeextractors.NwbSortingExtractor.write_sorting() method directly gives me the following error message stemming from h5py's io.write method: TypeError: No conversion path for dtype: dtype('<U11')

import spikeextractors as se

sorting = se.NeuroscopeMultiSortingExtractor(folder_path='D:\sample_data\sorted')
se.NwbSortingExtractor.write_sorting(sorting, 'D:\sample_data\sample_data.nwb')

By digging a bit deeper into the code, I found out that one can get rid of the error message by changing line 1066 in nwbextractors.py of the spikeextractors package from

nwbfile.add_unit(id=unit_id, spike_times=spkt, **unit_kwargs)

to

nwbfile.add_unit(id=1).

The resulting NWB file would then not contain any spike times, so this is certainly not a real fix.

The nwbfile.add_unit() method in turn is part of the pynwb package.

[bendichter]

@wuffi The Converter object is for combining data from multiple sources, e.g. electrophysiology and behavior. If all you need is to convert electrophysiology data, it might be more complexity than you really need. Your snippet here:

import spikeextractors as se

sorting = se.NeuroscopeMultiSortingExtractor(folder_path='D:\sample_data\sorted')
se.NwbSortingExtractor.write_sorting(sorting, 'D:\sample_data\sample_data.nwb')

is what I would recommend.

A fix will need to be made to the spikeextractors library, so I'd recommend raising an issue there (though we are active there too and we'd be happy to help resolve it).

h5py recently changed the way they handle strings with their recent switch from 2.x to 3.x, and I suspect that might be what is causing problems here. Could you try downgrading h5py to a 2.x version?

[bendichter]

Also, it would help if you could copy/paste the entire traceback

[wuffi]

Hi Ben, Thanks for your hints. Since I would also like to include behavior in my NWB file, going beyond the functionality provided by the spikeextractors package would probably make sense. I ran the spikeextractors part of the code on Linux and it's working fine there. That's why I assume it's a problem of my installation on Windows. The version of h5py on Windows would be 2.10 Trying to reproduce the traceback now gives me a new error message: KeyError: "'core' not a namespace"

Traceback (most recent call last):

  File "C:\Users\user\Documents\convert_sorting_to_nwb.py", line 8, in <module>
    import spikeextractors as se

  File "C:\Users\user\AppData\Local\Continuum\anaconda2\envs\nwb_conversion_env\lib\site-packages\spikeextractors\__init__.py", line 10, in <module>
    from .extractorlist import *

  File "C:\Users\user\AppData\Local\Continuum\anaconda2\envs\nwb_conversion_env\lib\site-packages\spikeextractors\extractorlist.py", line 11, in <module>
    from .extractors.nwbextractors.nwbextractors import NwbRecordingExtractor, NwbSortingExtractor

  File "C:\Users\user\AppData\Local\Continuum\anaconda2\envs\nwb_conversion_env\lib\site-packages\spikeextractors\extractors\nwbextractors\__init__.py", line 1, in <module>
    from .nwbextractors import NwbRecordingExtractor, NwbSortingExtractor

  File "C:\Users\user\AppData\Local\Continuum\anaconda2\envs\nwb_conversion_env\lib\site-packages\spikeextractors\extractors\nwbextractors\nwbextractors.py", line 14, in <module>
    import pynwb

  File "c:\users\user\downloads\pynwb\src\pynwb\__init__.py", line 260, in <module>
    from . import io as __io  # noqa: F401,E402

  File "c:\users\user\downloads\pynwb\src\pynwb\io\__init__.py", line 1, in <module>
    from . import base as __base

  File "c:\users\user\downloads\pynwb\src\pynwb\io\base.py", line 1, in <module>
    from .core import NWBContainerMapper

  File "c:\users\user\downloads\pynwb\src\pynwb\io\core.py", line 9, in <module>
    from pynwb.file import NWBFile

  File "c:\users\user\downloads\pynwb\src\pynwb\file.py", line 13, in <module>
    from .base import TimeSeries, ProcessingModule

  File "c:\users\user\downloads\pynwb\src\pynwb\base.py", line 9, in <module>
    from .core import NWBDataInterface, MultiContainerInterface, NWBData

  File "c:\users\user\downloads\pynwb\src\pynwb\core.py", line 37, in <module>
    class NWBContainer(NWBMixin, Container):

  File "c:\users\user\downloads\pynwb\src\pynwb\__init__.py", line 131, in _dec
    __TYPE_MAP.register_container_type(namespace, neurodata_type, cls)

  File "C:\Users\user\AppData\Local\Continuum\anaconda2\envs\nwb_conversion_env\lib\site-packages\hdmf\utils.py", line 561, in func_call
    return func(args[0], **pargs)

  File "C:\Users\user\AppData\Local\Continuum\anaconda2\envs\nwb_conversion_env\lib\site-packages\hdmf\build\manager.py", line 793, in register_container_type
    spec = self.__ns_catalog.get_spec(namespace, data_type)    # make sure the spec exists

  File "C:\Users\user\AppData\Local\Continuum\anaconda2\envs\nwb_conversion_env\lib\site-packages\hdmf\utils.py", line 561, in func_call
    return func(args[0], **pargs)

  File "C:\Users\user\AppData\Local\Continuum\anaconda2\envs\nwb_conversion_env\lib\site-packages\hdmf\spec\namespace.py", line 314, in get_spec
    raise KeyError("'%s' not a namespace" % namespace)

KeyError: "'core' not a namespace"

A similar problem is mentioned on https://github.com/NeurodataWithoutBorders/pynwb/issues/1058.

[bendichter]

Can you also provide the code you are running?

[rly]

import pynwb
...
KeyError: "'core' not a namespace"

It looks like you are using a local download of the pynwb source code. This error is most often due to not git cloning the pynwb repo with submodules. I.e. if you run git clone [pynwb url] instead of git clone --recurse-submodules [pynwb url] you will get the above error.

[wuffi]

Thanks, I'm afraid the --recurse-submodules flag it git clone was exactly what I was lacking. It's also mentioned on http://pynwb.readthedocs.io/en/stable/getting_started.html#install-latest-pre-release - I should have better read the manual.

@bendichter The code I had been running had been the same as above:

import spikeextractors as se

sorting = se.NeuroscopeMultiSortingExtractor(folder_path='D:\sample_data\sorted')
se.NwbSortingExtractor.write_sorting(sorting, 'D:\sample_data\sample_data.nwb')