higher CPU usage

[danieljdenman]

i'm setting up a Neuropixels 1.0 recording system. i've noticed that, when using 3+ probes connected to the same PXI card, the Open Ephys GUI uses a lot more CPU (even with no display sink (LFP or Probe Viewer) of any kind):

than SpikeGLX uses with the exact same hardware:

the CPU load with 3+ probes and an NI-6133 is too much when using the Open Ephys GUI - there are buffer overruns reported in the debug console and the GUI sometimes becomes unstable. when it does work, the PXie status light is, i think, "Purple: Combination of red and blue. Data is not read sufficiently fast by the PC. This indicates data loss". and then there are "RecordEngine timeout" messages in the debug console.

spikeGLX is able to acquire and write from the 4 probes and the NI-6133 stably, PXIe module status light remains blue.

system details: PXI Chassis: NI-1071 PXI Controller: NI-8301 (Thunderbolt) NI Multifunction IO: NI-6133 PC bus: Thunderbolt 3 PCIe adapter PC processor: Intel Xeon 224G, 4 Core, 8MB Cache, 3.5Ghz, 4.7Ghz Turbo PC RAM: 32GB DDR4 (4x8GB) PC disk: M.2 512GB PCIe NVMe Class 40

i'm not sure if this is expected from the Open Ephys GUI or not. should i try to build it myself instead of using the precompiled binares?

[jsiegle]

This is known issue with the current implementation of the RecordNode. Can you try following the instructions under the "Saving data" section on this page to record directly from the Neuropixels module? This will save data in the compressed NPX format (which needs to be extracted offline), but will allow you to record up to 16 probes at a time.

If you need to save the NI-6133 data as well, then instead of disabling the record thread entirely, you should just disable saving of all Neuropixels channels in the "REC" tab of the Neuropix-PXI module.

This approach is far from ideal, so @medengineer is working on a fix that involves turning the RecordNode into its own plugin that can be dropped into the signal chain. With the current architecture, every channel is copied to a separate data stream prior to saving, which ends up dramatically increasing CPU load with 384 x 2 x N channels, where N > 3. Avoiding this extra copying step should improve performance a lot.

[danieljdenman]

cool, thanks. NPX format allows the 4 probes to be recorded, and extraction worked perfectly (as far as i can tell). this worked for both disabling the record thread as well as disabling Neuropixels channel recording in the "REC" tab of the Neuropix-PXI module.

one note: it saves settings.xml in a folder in the directory at the top even with the record thread disabled, and no matter which directory is chosen directly on the Neuropix-PXI module. i think this is as expected, just notable in case people are looking for the settings.xml.

i'll keep an eye out for the update that implements that separate RecordNode plugin.

[jsiegle]

Yeah, settings.xml is still saved through the standard recording interface, so it's tricky to store it anywhere else.

[aacuevas]

Additionaly, the version now in the testing branch, which will be released as a binary soon, has already some improvements to help with Neuropixels. Not quite as drastic as the recordnode changes in progress, but it helps in some situations, in case you want to try it.