Fergil's Miniscope data notes

[weiglszonja]

Calcium (5 objects, ~3 GB)

• contains a documentation about the experiment and data format
• behavior data folder is empty (the behavior camera video(s) are missing)
• miniscope data folder has the following files:

C6-J588_Disc5_msCamAll_ffd.avi > This is a video of all the miniscope ‘Snippets’ (short videos capturing each trial), concatenated together into a single video. The first frame of every snippet is deleted (hence ‘ffd = first frame deleted) When I open this file in OpenCV, the number of frames are 10262, and the shape of a frame is (240, 376, 3) uint8 dtype.

C6-J588_Disc5_msCamComb_MC.mat > motion corrected video, when opening in MATLAB it has "Mr_8bit" named struct in it with 240 x 376 x 10262 uint8 values.

C6-J588_Disc5_msCamComb_MC_curated_20220605_1722.mat > this file should contain the calcium data, after cells have been identified by CNMF_E, and traces extracted. When I open this file in MATLAB I can see the "neuron" object however is has a different structure compared to what is expected from the readme:

Since I don't see struct named ‘C’ that would contain the traces of each cell throughout the task.

C6-J588_Disc5_timestampsAllCumulData.mat > msCamTrialTimestamps defines the first and last frame for each trial. They provided which were reward and shock trials: CS-Reward trails: [1, 3, 6, 9, 10, 12, 13, 15, 16, 20, 23, 24, 27, 28, 30], CS-Shock trails: [2, 4, 5, 7, 8, 11, 14, 17, 18, 19, 21, 22, 25, 26, 29] So we know for each frame whether it was a reward or shock trial. > trials table

There are other columns in the file which are not defined in the document:

@CodyCBakerPhD I'm not sure how standard this format is, I expected to be more similar to this, just thinking whether this interface will live in neuroconv or be a custom interface here.

[CodyCBakerPhD]

@CodyCBakerPhD I'm not sure how standard this format is, I expected to be more similar to this, just thinking whether this interface will live in neuroconv or be a custom interface here.

I believe this is externally parsed Miniscope - the native format I thought was the collection of .avi files plus some kind of header, but not MATLAB files that extracted that into arrays

When I open this file in MATLAB I can see the "neuron" object however is has a different structure compared to what is expected from the readme:

(6,1) arrays that have that appearance are the hallmark of MATLAB special encoding schemes. Normally you only see that when reading the .mat file in Python, this is my first time seeing it in MATLAB itself.

But if I had to guess it means that file was written in a certain version of MATLAB and your version is not compatible with it for whatever reason.

What version of MATLAB did you use to try reading it? Can we ask if they knew what version of MATLAB was used to write it?

[weiglszonja]

You're probably right, because it shows up like this in Python:

I'm using MATLAB_R2022b, but I opened MATLAB 5.0 MAT-files before that showed up "normally". (Actually all the files they have are MATLAB 5.0)

[weiglszonja]

TODO: for this issue try https://github.com/zhoupc/CNMF_E/tree/master/python_wrapper

[weiglszonja]

Created a helper function in MATLAB to load such file that was produced CNMF_E (also adding that folder to the path so MATLAB too can see the 'neuron' struct) and save it in a new file that can be read by Python:

% This is a helper function for repacking a .mat file produced by CNMF_E
% The 'neuron' struct is a Matlab class called Sources2D, so they cannot be directly loaded into Python for further analysis.

function save_neuron(CNMF_E_folder_path, source_filepath, destination_filepath)

    % Add CNMF_E folder to MATLAB search path
    addpath(CNMF_E_folder_path);

    % Load the file
    load(source_filepath, 'neuron');

    neuron = struct(neuron);

    % Save the 'neuron' struct in v7 format
    save(destination_filepath, '-v7', '-struct', 'neuron');

end

[weiglszonja]

TODO: check if segmentation output is compatible with our current CNMF_E extractor