Memmapfile for matlab

[kouichi-c-nakamura]

What is this PR about?

I modified load_open_ephys_data.m to support partial loading of CONTINUOUS data using datapoint indices using memmapfile. EVENT or SPIKE data are not supported.

Partial loading can be particularly useful when the data is very long (the file is big).

How to validate the change?

You'll need an Open Ephys *.continuous file.

Whole data

% filname should be a valid file path to a '*.continuos' file

[A.data, A.timestamps, A.info] = load_open_ephys_data(filename);

[B.data, B.timestamps, B.info] = load_open_ephys_data(filename,'Indices',[]);

assert(isequal(A,B)) % B should be identical to A

Partial loading

ind = [300:3300,50000:60000]; % non-consecutive, two parts

[C.data, C.timestamps, C.info] = load_open_ephys_data(filename,'Indices',ind);

assert(isequal(A.data(ind),C.data))

assert(isequal(A.timestamps(ind),C.timestamps))

assert(isequal(A.info.header,C.info.header))

assert(isequal(A.info.ts([1:4,49:59]),C.info.ts))

assert(isequal(A.info.nsamples([1:4,49:59]),C.info.nsamples))

assert(isequal(A.info.recNum([1:4,49:59]),C.info.recNum))

Near end of file

L = length(A.data)

[D.data, D.timestamps, D.info] = load_open_ephys_data(filename,'Indices',L-1023:L);

assert(isequal(A.data(L-1023:L),D.data))

assert(isequal(A.timestamps(L-1023:L),D.timestamps))

assert(isequal(A.info.header,D.info.header))

assert(isequal(A.info.ts(end),D.info.ts))

assert(isequal(A.info.nsamples(end),D.info.nsamples))

assert(isequal(A.info.recNum(end),D.info.recNum))

Issues

• EVENT or SPIKE data are not supported for partial loading
• Version < 0.2 has not been tested
• Corrupted file check code was copied underneath partial loading, but not tested. Not sure if it works.

[aacuevas]

Very useful addition, Thanks!

[kouichi-c-nakamura]

Wow, thank you for the quick merging.

I haven't tested the performance for really big data, but partial loading should be memory-efficient and probably faster than loading the whole thing and extracting subset.

I hope people love it.

Best, Kouichi