Data loss while streaming through LSL

[Gnarlywhale]

I am currently running pupil labs in conjunction with a motion capture setup through Lab Streaming Layer for data synchronization.

We've noticed that we get spikes of missing data when streaming eye positions. The data loss occurs in the data recorded by LSL as well as the raw pupil recordings (as exported in Pupil Player). For the most part streaming is fine, with the difference between time-stamps falling between an expected range. However, for roughly every 10 seconds of data streaming, we encounter a large delay between subsequent recorded frames, ranging between ~100ms up to ~1 sec.

We've also noticed that these large delays seem to sync up with periods where the "CPU" visualization at the top of each eye window is dropping extremely low. There seems to be no issue of significant delays when the pupil labs relay plugin is not turned on.

For example:

The top row of plots shows the difference between subsequent timestamps as received by LSL for each eye, the bottom row showing the difference in timestamps as exported by pupil capture. Any guidance into rectifying the discrepancy between the delays missing between LSL and the pupil recordings, and in getting rid of the delay spikes all together is greatly appreciated.

This is on a Windows 10 machine.

[mkassner]

are you using 2d or 3d mode?

[Gnarlywhale]

3d mode

[mkassner]

Could you try to reproduce with 2d mode? I think this could be a solver lock bug in 3d mode.

[Gnarlywhale]

Here's the result of testing with 2d (worst case of 10 trials):

Again the top row is the time stamp differences as recorded by LSL, and the bottom row as exported by pupil capture.

In this case there doesn't seem to be any missing frames in the pupil export, and a ~65ms delay in the LSL data. Out of 10 trials, 2 had delays ~65ms and the rest were < 20ms.

[Gnarlywhale]

Adjusting the pupil camera frame rates also impacted observed delays, with an increase in framerate corresponding with lag spikes of increased magnitude.

[Gnarlywhale]

Running just pupil labs with no streaming relay in 3D mode still does indeed yield delays. Here are the timestamp differences for one eye:

The largest delay for this trial was 653 msec between frames, vs. an average of 8.3 msec (expected).

I've checked and each of the shown delays coincides with a new model_id, with the 653 msec delay accompanying a jump from model 4 to model 10. Note, not all model increments caused a significant delay.

[mkassner]

Ok. This data tells me that:

• 2d mode timestamps generated by Pupil are fine.
• 3d mode we have the lock issue that creates basically a freeze period in which a number of frames are lost.

I dont know how timestamps are generated in LSL. @papr do you have any insight?

[Deepucdactvm]

I am currently working on a project involving EEG data acquisition using LSL (Lab Streaming Layer). I’ve encountered an issue where data seems to be lost during the streaming process, and I need some guidance on how to address this problem.