Trying to convert data stored in a Mat File to Binary for Kilosort4, receiving Error concerning empty array

[AidanFishenden]

I am Trying to use Kilosort 4 to sort through Electrophysiology data stored in a collection of mat files. Each file contains the data from a single channel, displayed as a 1 by 8556636 double variable in MATLAB. Highest value in the data is 0.0108, lowest value is -0.0107. There are a total of 32 channels on the probe. Any attempt I have made to convert these files, either combined into a 32 by 8556636 matrix or individually, to binary for Kilosort 4 has resulted in the following error when I attempt to run Kilosort.

kilosort.gui.sorter: Encountered error in run_kilosort: Traceback (most recent call last): File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\kilosort\gui\sorter.py", line 114, in run st, tF, Wall0, clu0 = detect_spikes( File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\kilosort\run_kilosort.py", line 521, in detect_spikes st0, tF, ops = spikedetect.run(ops, bfile, device=device, progress_bar=progress_bar) File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\kilosort\spikedetect.py", line 204, in run ops['wPCA'], ops['wTEMP'] = extract_wPCA_wTEMP( File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\kilosort\spikedetect.py", line 72, in extract_wPCA_wTEMP model = TruncatedSVD(n_components=ops['settings']['n_pcs']).fit(clips) File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\decomposition_truncated_svd.py", line 208, in fit self.fit_transform(X) File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\utils_set_output.py", line 313, in wrapped data_to_wrap = f(self, X, *args, *kwargs) File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\base.py", line 1473, in wrapper return fit_method(estimator, args, kwargs) File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\decomposition_truncated_svd.py", line 228, in fit_transform X = self._validate_data(X, accept_sparse=["csr", "csc"], ensure_min_features=2) File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\base.py", line 633, in _validate_data out = check_array(X, input_name="X", check_params) File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\utils\validation.py", line 1087, in check_array raise ValueError( ValueError: Found array with 0 sample(s) (shape=(0, 61)) while a minimum of 1 is required by TruncatedSVD.

Traceback (most recent call last):

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\kilosort\gui\sorter.py", line 114, in run

st, tF, Wall0, clu0 = detect_spikes(

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\kilosort\run_kilosort.py", line 521, in detect_spikes

st0, tF, ops = spikedetect.run(ops, bfile, device=device, progress_bar=progress_bar)

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\kilosort\spikedetect.py", line 204, in run

ops['wPCA'], ops['wTEMP'] = extract_wPCA_wTEMP(

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\kilosort\spikedetect.py", line 72, in extract_wPCA_wTEMP

model = TruncatedSVD(n_components=ops['settings']['n_pcs']).fit(clips)

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\decomposition_truncated_svd.py", line 208, in fit

self.fit_transform(X)

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\utils_set_output.py", line 313, in wrapped

data_to_wrap = f(self, X, *args, **kwargs)

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\base.py", line 1473, in wrapper

return fit_method(estimator, *args, **kwargs)

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\decomposition_truncated_svd.py", line 228, in fit_transform

X = self._validate_data(X, accept_sparse=["csr", "csc"], ensure_min_features=2)

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\base.py", line 633, in _validate_data

out = check_array(X, input_name="X", **check_params)

File "C:\Users\aidan-admin\anaconda3\envs\kilosort\lib\site-packages\sklearn\utils\validation.py", line 1087, in check_array

raise ValueError(

ValueError : Found array with 0 sample(s) (shape=(0, 61)) while a minimum of 1 is required by TruncatedSVD.

Does anyone know how to fix this error? I am also including the matlab code that I used to convert the file.

% Define file paths matFilePath = 'E:\Dataforkilosort\OneDrive_1_6-15-2023\1_ftc\Pen1.mat'; % Replace with your actual .mat file path binaryFilePath = 'E:\Dataforkilosort\OneDrive_1_6-15-2023\1_ftc\Pen1B2.dat'; % Path for the binary file to be created

% Load the .mat file matData = load(matFilePath);

% Extracting variable eeg_data = allDataMatrix;

% Checking dimensions [nChannels, nSamples] = size(eeg_data); disp(['Number of Channels: ', num2str(nChannels)]); disp(['Number of Samples: ', num2str(nSamples)]);

% Transpose the data to match Kilosort's expected format (n_samples x n_channels) eeg_data = eeg_data'; % Kilosort expects (n_samples, n_channels)

% Save data to a binary file fileID = fopen(binaryFilePath, 'wb'); fwrite(fileID, eeg_data, 'float32');
fclose(fileID);

[jacobpennington]

Hello,

The binary file should have shape (n_samples, n_channels) in C-order (row-major), it sounds like you're doing (n_channels, n_samples).

Also make sure you're setting dtype = float32 when running Kilosort, the default is int16. You may also need to use the scale parameter to get the data into the expected ~100ish order of magnitude, it looks like that would be scale = 1e4 for you. Try it without scaling first, but if you continue to get errors that would be the next thing to try.

[AidanFishenden]

Adjusting the scaling worked perfectly, thank you!