BlackrockNeurotech / Python-Utilities

A collection of scripts for loading and manipulating Blackrock Microsystems datafiles.
27 stars 16 forks source link

Ptp fix #22

Closed dkluger closed 1 year ago

dkluger commented 1 year ago

This is a pull request for merging in changes to the class NsxFile to allow for files with 1-sample segments in continuous data. This allows for PTP timestamping of continuous data. The method getaligneddata() allows for continuous data frame copies/removals to allow for PTP-timestamped data to be aligned to the "true" sampling rates (30 kS/s, 10 kS/s, etc). This allows data recorded from a Gemini NSP and Gemini Hub to be aligned for analysis after alignment.

cboulay commented 1 year ago

I made quite a few changes in 2f06cf6. I'm sure I'm forgetting some, but here's what I can remember, in no particular order:

The data loading is now lazy, and as such is no longer deferred until after header inspection. This simplifies logic and lazy-loading drastically speeds up function return time. The data won't be loaded into memory until the user slices the data. If they are only slicing specific segments, or using a rolling-window for analysis, this will lead to greater performance and few memory bottlenecks.

However, using either zeropad or force_srate might cause the data to be loaded into memory because the returned array is modified with data not on disk, so it's impossible to do lazy-loading.

@dkluger , this probably needs a re-review now, but I'll mark it as ready.