Error during TDT Recording Extractor

[kedoxey]

This error appears during the TDT recording extractor when converting to NWB format within NWB Guide and NeuroConv

Traceback (most recent call last):
  File "flask/app.py", line 1484, in full_dispatch_request
  File "flask/app.py", line 1469, in dispatch_request
  File "flask_restx/api.py", line 404, in wrapper
  File "flask/views.py", line 109, in view
  File "flask_restx/resource.py", line 46, in dispatch_request
  File "namespaces/neuroconv.py", line 70, in post
  File "manageNeuroconv/manage_neuroconv.py", line 414, in get_metadata_schema
  File "manageNeuroconv/manage_neuroconv.py", line 377, in instantiate_custom_converter
  File "neuroconv/nwbconverter.py", line 80, in __init__
    self.data_interface_objects = {
  File "neuroconv/nwbconverter.py", line 81, in <dictcomp>
    name: data_interface(**source_data[name])
  File "neuroconv/datainterfaces/ecephys/tdt/tdtdatainterface.py", line 40, in __init__
    super().__init__(
  File "neuroconv/datainterfaces/ecephys/baserecordingextractorinterface.py", line 38, in __init__
    self.recording_extractor = self.get_extractor()(**source_data)
  File "spikeinterface/extractors/neoextractors/tdt.py", line 34, in __init__
  File "spikeinterface/extractors/neoextractors/neobaseextractor.py", line 244, in __init__
  File "spikeinterface/core/baserecording.py", line 41, in __init__
  File "spikeinterface/core/baserecordingsnippets.py", line 25, in __init__
TypeError: data type "<class 'int'>" not understood

[CodyCBakerPhD]

@kedoxey If you still have it, the similar traceback that you got from the command line call also displayed the final line in SI with the error - can you copy that over? It had something to do with setting a numpy dtype

For context, I the latest versions of SI/neo work fine on current known example TDT data, but something subtle changes with @kedoxey's data

This type of error stems from a call of the form

numpy.dtype(str(type(1)))

implying some instance/type did not get parsed as expected (casted a string on the type rather than the value or something similar)

[kedoxey]

Error trace from command line:

Traceback (most recent call last):
  File "/Users/katedoxey/Desktop/research/projects/CRCNS/data/nwb_conversion/convert_tdt.py", line 14, in <module>
    interface = TdtRecordingInterface(folder_path=tdt_dir, gain=1.0, verbose=False)
  File "/Users/katedoxey/miniconda3/lib/python3.9/site-packages/neuroconv/datainterfaces/ecephys/tdt/tdtdatainterface.py", line 40, in __init__
    super().__init__(
  File "/Users/katedoxey/miniconda3/lib/python3.9/site-packages/neuroconv/datainterfaces/ecephys/baserecordingextractorinterface.py", line 38, in __init__
    self.recording_extractor = self.get_extractor()(**source_data)
  File "/Users/katedoxey/miniconda3/lib/python3.9/site-packages/spikeinterface/extractors/neoextractors/tdt.py", line 34, in __init__
    NeoBaseRecordingExtractor.__init__(
  File "/Users/katedoxey/miniconda3/lib/python3.9/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py", line 244, in __init__
    BaseRecording.__init__(self, sampling_frequency, chan_ids, dtype)
  File "/Users/katedoxey/miniconda3/lib/python3.9/site-packages/spikeinterface/core/baserecording.py", line 41, in __init__
    BaseRecordingSnippets.__init__(
  File "/Users/katedoxey/miniconda3/lib/python3.9/site-packages/spikeinterface/core/baserecordingsnippets.py", line 25, in __init__
    self._dtype = np.dtype(dtype)
TypeError: data type "<class 'int'>" not understood

[h-mayorquin]

@CodyCBakerPhD this does not look like a spikeinterface problem. It seems that the at some point in the chain the dtype is serialized to string str(int) and then is not unserliazed correctly.

Probably something with the json at the communication between neuroconv and the guide.

[h-mayorquin]

This is really confusing because these lines:

https://github.com/SpikeInterface/spikeinterface/blob/bd89c998afb29c204167a4f7aff360a1cd631da5/src/spikeinterface/extractors/neoextractors/neobaseextractor.py#L244-L245

Before calling the __init__ of BaseRecording on neobasextractors wher the error of @kedoxey is raised the dtype should already have been casted to dtype.

I don't think there is a way where the output of dtype can be a str(int). Is there?

So I think that there is something outside of spikeinterface when handling this case of neo maybe?

https://github.com/NeuralEnsemble/python-neo/blob/5d66cced6f85129e715d9f257e6240b25cfcb0cf/neo/rawio/tdtrawio.py#L253-L256

@kedoxey could you share your data just to be 100 % this is the case?