Slice out a recording segment and resave in open ephys

[nmtimme]

Hello

My group previously posted an issue (#25) about how to trim off the end of a recording and resave it. We've found some other recordings with other bad segments (e.g., a brief time discontinuity about 1 second into the recording). How could we select part of a recording and keep only that part while also properly modifying the support files (e.g., all_channels.events, Continuous_Data.openephys, messages.events, and settings.xml)? In other words, suppose I had a recording that was 1000 seconds long. I want to be able to pull out seconds 5-995 in the .continuous files and the support files and resave it all in open ephys format as a new data set. I would appreciate any guidance on how to do this or on existing tools that could accomplish this goal. Thank you!

~Nick

[jsiegle]

Hi Nick – You can use the same method I posted previously to trim both the beginning and end of the recording: https://github.com/open-ephys/open-ephys-python-tools/issues/25#issuecomment-1764962078

Each record is 1024 samples (1024/sample_rate seconds), and 2070 bytes. So 5 seconds would be equal to 146 records. You can then trim the file like so:

record_size_in_bytes = 2070
data = np.memmap('100_CH1.continuous', mode='r', dtype='uint8')
total_bytes = len(data)
bytes_to_cut = record_size_in_bytes * 146
data[bytes_to_cut:total_bytes-bytes_to_cut].tofile('100_CH1_trimmed.continuous')

You do not need to make any changes to the additional files, since the sample numbers will still be aligned after trimming. There may be a few events with sample numbers that fall outside the range of the continuous data, which can be ignored for analysis.

This approach can be used to repair previously recorded sessions that have corrupted segments. For future data collection, I strongly recommend upgrading to the latest version of the software (0.6.6), which will prevent these issues from happening again.

[jsiegle]

I forgot about the 1024-byte header, which needs to be extracted an replaced. So the last line should actually be replaced with this:

header_bytes = data[:1024]
trimmed_data = np.concatenate((header_bytes, data[1024+bytes_to_cut:total_bytes-bytes_to_cut]))
trimmed_data.tofile('100_CH1_trimmed.continuous')

[nmtimme]

@jsiegle ,

Thank you! I implemented this and it appears to work perfectly.

I do have one other question. What do we need to do to realign the time once we have spike sorted data. Suppose, for instance, that we have a 10 minute recording and we trim off the first 2 seconds from the .continuous files. Then, we spike sort the data with kilosort3 (for instance). If a spike appeared at the 60 second mark in the untrimmed recording, will it show up at the 58 second mark in the trimmed recording? Also, the event times would should basically be reduced by 2 seconds and any events with negative times should be ignored, right? Basically, I'm curious how (1) this trimming will alter the time stamps in the data and (2) how any alterations to the time stamps will be propagated through spike sorting. I'd appreciate any advice you can provide on this topic.

Thank you!

~Nick

[jsiegle]

Hi Nick – Kilosort "spike times" are actually sample numbers relative to the start of the data file. So you don't need to do any correction if you sort the data after the file has been trimmed. You can find the global timestamp (in seconds) for each spike by using these sample numbers to index into the timestamps of the continuous data.

[nmtimme]

Hi Josh,

Thank you for clarifying! This works well.

~Nick