Can read_tdt() handle .sev files?

[mberns-ru]

Hi! Currently, I am trying to load data into a recording object like this: full_raw_rec = si.read_tdt(folder_path=tdt_folder, stream_name = "b'Wav1'")

where _tdtfolder has the following structure:

We use a 64-channel probe, and each channel is exported into a .sev file. However, when I pass in _tdtfolder to read_tdt(), I get the following error:

My understanding - based on Issue #2425 - is that read_tdt() is looking for a .tev file. When I pass in that file directly, it does work; however, our .tev file doesn't include the 64 channels that we want, which are stored in 64 separate .sev files. It seems that Neo's TdtRawIO function does support .sev files.

I'm unsure how to go about loading these .sev files into a single recording object. I was wondering if anyone could provide some guidance? I apologize if this is a silly question; I'm new to this!

[zm711]

Maybe related: https://github.com/NeuralEnsemble/python-neo/issues/1286

Is this a newer style set of .sev files. If so we may want to open an issue on Neo and if you can supply some sample files or a folder we can work on fixing that over there. I don't know tdt at all, but with sample files I can at least try to see what is going on.

[h-mayorquin]

@victornovakov does that mean that https://github.com/SpikeInterface/spikeinterface/issues/2825 worked for you? Can we close it?

[victornovakov]

@h-mayorquin yes it can be closed. Sorry I forgot to follow up on the issue.

[h-mayorquin]

No worries, it is good that you asked so when somebody googles it might land them there.

[mberns-ru]

@zm711 Sorry for the delay! I commented on that Neo post with a sample dataset.

[zm711]

Hi @mberns-ru!

So your error is actually due to the fact that your dataset is a single TDT block rather than a multi-block dataset. So what you need to do is:

tdt_folder = r'Path/to/folder'
full_raw_rec = si.read_tdt(folder_path=tdt_folder / 
'A_Only_Test_Ephys_Passive_HL-240514-091335_J_M41_Tail-240523-094748.Tbk' ,
stream_name = "b'Wav1'")

Notice that I took your folder path and added on one file inside the folder path. For single-block you have to input any file to get the directory. For multi-block you have to put in the root directory.

I think this is actually pretty confusing. Is there a reason for this @samuelgarcia (you wrote the IO originally)?

[mberns-ru]

Hey @zm711 ! Thanks for your help. I tried that code as well as using the .tev file in the folder, and it resulted in the same issue, where it only shows 1 channel:

Is this how it's showing up for you too?

It should be noted that our .tev file does not contain data for all 64 of our streams, only data from our behavior task. That data is only contained in our .sev files. I tried looking into the .tbk file a couple days ago, but I didn't have any software that allowed me to open it.

I'm working on a workaround right now that converts .sev files into .bin files, so hopefully I can just use si.read_bin() instead. But I haven't figured that out yet:

full_rec_block = tdt.read_block(tdt_folder, store="b'Wav1'", export='binary')
bin_path = Path(str(tdt_folder) + '\\binary_export.txt')
print(full_rec_block)
full_raw_rec = si.read_binary(file_paths=bin_path, num_channels = 64, sampling_frequency = samp_freq, dtype=bin_dtype, time_axis=1)

[zm711]

This is what I can see on my end:

Since I don't know tdt super well I might not be the best at knowing where to look for each thing, but you definitely have streams with 64 channels. Why did you choose b'Wav1' as your stream?

[mberns-ru]

@zm711 Whoa that worked! A little embarassing on my end... thank you so much!

[zm711]

No worries @mberns-ru :) Happy to help. Let us know if anything else comes up!

[mberns-ru]

@zm711 Hi, so sorry! I'm back with another issue:

After loading in the correct stream (the only 64-channel stream that doesn't generate a float type error), I've found that it isn't storing any channel location data. Is this in reference to the channel location on the probe? The remapping seems to look fine in the generated Pandas dataframe.

Again, thank you so, so much for helping!

[zm711]

TDT doesn't store channel locations so yep this refers to the map. What you need to do is something like:

raw_rec_w_probe = full_raw_rec.set_probe(probe)

For making a probe you can go to https://probeinterface.readthedocs.io/en/main/

Once you make the probe then you just attach it to the recording! We have most of the probes from Cambridge Neurotech already generated and connections between these probes and both the 32 and 64 channel headstages from Intan. Others may require you to build your own probe which explained in the documentation as well. Let us know if you don't understand the process!

[mberns-ru]

I think I was able to set it up correctly according to our channel map:

# Import probe
import probeinterface as pi
from probeinterface.plotting import plot_probe

manufacturer = 'cambridgeneurotech'
probe_name = 'ASSY-276-P-1'

probe = pi.get_probe(manufacturer, probe_name)
# Set wiring:
device_channel_indices = [
    8,4,12,5,22,6,21,17,13,14,28,1,30,25,29,20,52,53,54,61,49,41,38,36,62,57,60,44,37,46,45,33,
    2,3,9,7,11,10,16,15,19,18,24,23,27,26,32,31,35,34,40,39,43,42,48,47,51,50,56,55,59,58,64,63]
probe.set_device_channel_indices(device_channel_indices)

"""fig, ax = plt.subplots(figsize=(14, 10))
plot_probe(probe, ax=ax, with_device_index=True)
ax.set_xlim(-100, 900)
ax.set_ylim(-50, 300)
plt.show()"""

# Probe visualized as Pandas dataframe:
probe.to_dataframe(complete=True).loc[:, ["contact_ids", "shank_ids", "device_channel_indices"]]
full_raw_rec = full_raw_rec.set_probe(probe, group_mode="by_shank")

However, I now get this error on the very last line where it says the device_channel_indices do not match the channel count, despite the only difference being that one is an array of ints and the other is an array of strings:

[alejoe91]

device_channel_indices are indices, so they should be 0-based, not 1-based ;)

[mberns-ru]

@alejoe91 Thank you so much!