Merging multiple OpenEphys binary (continuous.dat) files

[saman-abbaspoor]

Hi all,

In my experiment, we record multiple binary files corresponding to different behavioral states during a session. I would like to merge them for data analysis/spike sorting. Can you suggest some of the most efficient ways to do this? I also read on one of the topics that .continuous files had 1024 samples as headers. Is this also the case for OpenEphys binary (.dat) files?

Thank you in advance, SAM

[jsiegle]

Hi Sam –

You can do this fairly easily using numpy.ndarray.tofile

The .dat files don't have a header, so you can load them efficiently using memory mapping. You will need to know the number of channels in the file to do this (check the structure.oebin file for this number).

import numpy as np
data = np.memmap(filename, dtype='int16')
data = np.reshape(data, (data.size // numChannels, numChannels)

Once you have the data loaded, you can write it to a file using:

f = open(newfilename, 'wb')
data.tofile(f)
f.close()

Now, you can add additional recordings to the same file by opening the file in "append" mode:

f = open(newfilename, 'a')
data2.tofile(f)
f.close()

Let me know if that works! Just keep in mind that this may be pretty slow if your data files are large.