Read Meta function optimization for Open Ephys

[vasilisazhukova]

Hello,

I am trying to process my data, which I recorded with Open Ephys, in notebook ''extract_raw_data_demo'. Meta data from Open Ephys are different from ones aquired with SpikeGLX. For example, I could't find anything in my structure.oebin file resemble to 'fileSizeBytes'. Could you please explain what this 'fileSizeBytes' is? If I use NP2.0, 'nSavedChans' will be 383, right? Also, in the case you have a version of Read_Meta function for Open Ephys, it would be helpful if you could include this code in UnitMatchPy. json loads meta data from Open Ephys.

Thank you. Best regards, Vasilisa

[EnnyvanBeest]

Dear Vasilisa,

fileSizeBytes should just be the size of the file (I believed it's called 'continuous.dat' in OpenEphys?) in bytes. I think something like this should do: n_elements = filos.path.getsize("/path/to/continuous.dat") nSavedChans should be 384 + nSync if you use normal settings. This would be ''num_channels" in your openEphys .oebin file.

Perhaps for now you assign these parameters manually, and I'll ask @Sam-Dodgson to add an OpenEphys case in the ReadMeta part of the code.

Best, Enny

[vasilisazhukova]

@EnnyvanBeest thank you for your rapid answer. what do you mean by 'nSync'? I have only 384 in total for ''num_channels".

When I leaft n_elements = os.path.getsize("/path/to/continuous.dat") n_channels_tot = 384

I received a very interesting error. Any suggestions in its debugging?

[EnnyvanBeest]

No other suggestions than the one provided by python: the error seems to suggest that idx is an array rather than a single value, and I don't know why it's doing this as the code (line 104 of extract_raw_data.py) should loop over single values. Sam is working with some example OpenEphys data we have to make this function compatible for OpenEphys users, so hopefully these errors will be solved for you soon.

For the number of channels: for SpikeGLX the number of channels is the number of recording channels (typically 384) + the number of 'sync' channels (typically one). If you don't use a sync channel, it may indeed be 384 (384+0).

[vasilisazhukova]

@EnnyvanBeest @Sam-Dodgson thank you very much.

[Sam-Dodgson]

Hi i have added a new notebook called extract_raw_data_demo_open_ephys.ipynb which should be able to handle open ephys data, please let me know if there is a problem

[vasilisazhukova]

@Sam-Dodgson thank you very much, but I don't see this file.

[Sam-Dodgson]

Hi, The file is on the Work In Progress branch, please let me know if there is a problem as I don't have access to a full open ephys data set! On 30 Oct 2024, at 10:05, vasilisazhukova @.> wrote: @Sam-Dodgson thank you very much, but I don't see this file. grafik.png (view on web) — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned. Message ID: @.>

[vasilisazhukova]

@Sam-Dodgson I mean I don't see the notebook you mentioned in the list of UnitMatchPy notebook. Could you please add it in this folder?

[EnnyvanBeest]

Sam meant it's in a different branch than the main. It can be found on the workinprogress branch: https://github.com/EnnyvanBeest/UnitMatch/tree/WorkInProgress/UnitMatchPy/Demo%20Notebooks.

Once you confirm it works we will pull it to the main branch. Thank!

[Sam-Dodgson]

Hello, I cant see what the error is in those screen shots as the Parallel function hides the actual error, if you run the function for a unit when there is no parallelisation e.g calling extract_a_unit(sample_idx, data, half_width, spike_width, n_channels, sample_amount) with the value for 1 specific unit. This should hopefully help find the error, if this is not possible we could try to find a way to send me the data which is not working and i can try to find the problem on my end. best, Sam On 31 Oct 2024, at 11:29, vasilisazhukova @.> wrote: Hello, thank you again for your help. I encountered an error I can't fix in the loop for avg_waveforms: error.in.avg_waveforms.PNG (view on web) image.png (view on web) — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned. Message ID: @.>

[vasilisazhukova]

@Sam-Dodgson , I forgot to set True for KS4_data, and it resulted in this error. Excuse me for bothering you. Good news that the code worked on my data! Only in the last line of the code "GoodUnits" should be replaced with "good_units" erd.save_avg_waveforms(avg_waveforms, KS_dirs[sid], GoodUnits= good_units[sid], extract_good_units_only = extract_good_units_only). For some data samples I had to change the peak width from 61 to 50. Now, the full UnitMatch pipeline works for my data. Thank you for your help!