LSL measuring actual latency of online EEG data

[sappelhoff]

Hi everyone, I am currently implementing a BCI system with realtime access to EEG data streamed from a BrainAmp via the LSL app "BrainAmpSeries". Currently, the performance of my system is not as expected and therefore, I wanted to measure the latency of the EEG data. This is how I conceptualize it:

• Latency B is easily measured by timing the call "pull_chunk"
• To measure latency A, we save chunks of online data extracted after a trigger signal and then after the recording, we try to align the online data with the offline data (recorded with LabRecorder) ... via the index of perfect alignment between the two data, we can get latency A (see also my implementation)
• The overall latency is obviously latency A plus latency B.

My results:

1. Latency B will be higher or lower depending on the size of the chunk that I am pulling.
2. Latency A is independent of latency B

My questions:

1. Is this the correct way to conceptualize latency in terms of the LSL?
2. Is the measuring performed in a proper way?
3. On which factors does latency A depend? --> Firewall? USB connection vs. connection via e.g., PCI card to the amplifier? ... Some irreducible latency?

I am currently at 65ms of latency for latency A, which is really a lot - but I cannot pin down the reason for it. Help would be greatly appreciated. For more information, please just ask me, or check out the repository I have made for this issue repository I have made for this issue.

[sappelhoff]

I have now found several resources on this issue. They were just buried too deeply for me to find before opening this issue:

TL;DR

1. Several people have experienced latency with the BrainAmpSeries (or the RDA) app
2. This latency is not due to the LSL apps, but rather stems from the hardware or drivers
3. The latency, or "age of samples" as @otraupe calls them is the time between an event happening at an electrode and that event being timestamped as it reaches LSL
4. The age of samples seems to be around 40-70ms depending on the exact setup
5. A quick solution for an online system would be to wait an amount t after an event before pulling the online data. With t being a well tested age of samples on the used system.
6. This is still a drawback considering that the wanted online data will arrive delayed and with a jitter.

Resources

• Issue #43 contains several tests for the age of samples:
• Connected to issue #43, there is an exchange in the mailling list from 2015
• Related to the tests, there is a comprehensive set of tests by @otraupe available here as announced in the mailing list in 2017

I suggest to incorporate this information into the Wiki as a note of caution --> this behavior is not something that a user might think about in the beginning.

[dmedine]

65ms seems a little high, but not much. I believe the absolute latency of the BrainAmp typically hovers around 45-50ms on average (though I don't know the details as to why, probably this is a fixed hardware issue) and then after some blocking in LSL (32/5000 seconds) is another 6ms, add a few more for the OS to do its thing and you are maybe at around 55ms or so? This is on the order of 65ms.

1. To me, latency is irrelevant (as long as it is known) unless the application is a real-time one (like a BCI). Beyond that, everything depends on your needs and your setup. The absolute latency of LSL itself is very low, even over a network (https://sccn.ucsd.edu/~mgrivich/LSL_Validation.html). Hardware always has latency and must be measured. Certain devices are super-jittery (especial HCI devices like keyboard and mouse events) due to how the OS prioritizes polling.

2. I'm unfortunately not in a position to evaluate your methods, but your results seem to be correct (for what it's worth).

3. A most likely cannot be changed. The latencies on the hardware side are

1. the time it takes to fill the hardware buffer with past samples, 2. the length of the AD lowpass filter (usually fairly long), 3. USB associated latencies. There's nothing you can do to change these.

You should contact Brain Products technical support for more detailed explanation of the hardware (techsup@brainproducts.com), but I do know that they don't recommend BrainAmp for low-latency applications.

On 2/1/2018 9:53 AM, Stefan Appelhoff wrote:

Hi everyone, I am currently implementing a BCI system with realtime access to EEG data streamed from a BrainAmp via the LSL app "BrainAmpSeries". Currently, the performance of my system is not as expected and therefore, I wanted to measure the latency of the EEG data. This is how I conceptualize it:

lsl_latency https://user-images.githubusercontent.com/9084751/35669014-feb807f4-0733-11e8-927a-09e7e4344130.png

• Latency B is easily measured by timing the call "pull_chunk"
• To measure latency A, we save chunks of online data extracted after a trigger signal and then after the recording, we try to align the online data with the offline data (recorded with LabRecorder) ... via the index of perfect alignment between the two data, we can get latency A (see also my implementation https://github.com/sappelhoff/BrainAmp_LSL_latency)
• The overall latency is obviously latency A plus latency B.

My results:

1. Latency B will be higher or lower depending on the size of the chunk that I am pulling.
2. Latency A is independent of latency B

My questions:

1. Is this the correct way to conceptualize latency in terms of the LSL?
2. Is the measuring performed in a proper way?
3. On which factors does latency A depend? --> Firewall? USB connection vs. connection via e.g., PCI card to the amplifier? ... Some irreducible latency?

I am currently at 65ms of latency for latency A, which is really a lot

• but I cannot pin down the reason for it. Help would be greatly appreciated. For more information, please just ask me, or check out the repository I have made for this issue repository I have made for this issue https://github.com/sappelhoff/BrainAmp_LSL_latency.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/sccn/labstreaminglayer/issues/269, or mute the thread https://github.com/notifications/unsubscribe-auth/ADch7i9Dz_x4o1w1W2j5aZgmMcWfSfiuks5tQXuigaJpZM4R1QPS.

[otraupe]

David, you overlook that a) these 67 ms pertain to the RDA interface of the BrainVision Recorder, which adds another ~10 ms to the cascade, at least with the employed software versions, and b) we are using the wifi-based MOVE transmission, which adds another ~5 ms.

You can find my detailed conclusions here: http://bemobil.bpn.tu-berlin.de/wiki/doku.php?id=software:lsl-test#conclusions

[sappelhoff]

Thank you @dmedine for your statements! And of course @otraupe for these wonderful tests.

@dmedine do you happen to have a reference to an article or similar where BrainProducts state that they do not recommend the BrainAmp for latency critical BCIs?

Finally, for documentation purposes, here is also an update of my figure to incorporate what I have learned through the resources cited above:

LSL_latency.pdf

[dmedine]

Oh. I misunderstood. I thought that this was going directly into the BrainAmp connector app.

On 2/2/2018 12:09 PM, otraupe wrote:

David, you overlook that a) these 67 ms pertain to the RDA interface of the BrainVision Recorder, which adds another ~10 ms to the cascade, at least with the employed software versions, and b) we are using the wifi-based MOVE transmission, which adds another ~5 ms.

You can find my detailed conclusions here: http://bemobil.bpn.tu-berlin.de/wiki/doku.php?id=software:lsl-test#conclusions

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/sccn/labstreaminglayer/issues/269#issuecomment-362558157, or mute the thread https://github.com/notifications/unsubscribe-auth/ADch7qJhrQvlDxeAQG7SdLrj0c1SHON6ks5tQu0CgaJpZM4R1QPS.

[dmedine]

Unfortunately no, I do not. That might be something nice to have, though.

On 2/2/2018 12:12 PM, Stefan Appelhoff wrote:

Thank you @dmedine https://github.com/dmedine for your statements! And of course @otraupe https://github.com/otraupe for these wonderful tests.

@dmedine https://github.com/dmedine do you happen to have a reference to an article or similar where BrainProducts state that they do not recommend the BrainAmp for latency critical BCIs?

Finally, for documentation purposes, here is also an update of my figure to incorporate what I have learned through the resources cited above:

LSL_latency.pdf https://github.com/sccn/labstreaminglayer/files/1689094/LSL_latency.pdf

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sccn/labstreaminglayer/issues/269#issuecomment-362558609, or mute the thread https://github.com/notifications/unsubscribe-auth/ADch7sriIZ-xKIB445EPma4rm3jmprQAks5tQu2GgaJpZM4R1QPS.

[otraupe]

Stefan, you might want to contact Thorsten Zander about that. If anyone know, it is him. Likely, there is a paper by him, you could cite.

On 02.02.2018 14:51, David Medine wrote:

Unfortunately no, I do not. That might be something nice to have, though.

On 2/2/2018 12:12 PM, Stefan Appelhoff wrote:

Thank you @dmedine https://github.com/dmedine for your statements! And of course @otraupe https://github.com/otraupe for these wonderful tests.

@dmedine https://github.com/dmedine do you happen to have a reference to an article or similar where BrainProducts state that they do not recommend the BrainAmp for latency critical BCIs?

Finally, for documentation purposes, here is also an update of my figure to incorporate what I have learned through the resources cited above:

LSL_latency.pdf

https://github.com/sccn/labstreaminglayer/files/1689094/LSL_latency.pdf

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub

https://github.com/sccn/labstreaminglayer/issues/269#issuecomment-362558609,

or mute the thread

https://github.com/notifications/unsubscribe-auth/ADch7sriIZ-xKIB445EPma4rm3jmprQAks5tQu2GgaJpZM4R1QPS.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sccn/labstreaminglayer/issues/269#issuecomment-362591620, or mute the thread https://github.com/notifications/unsubscribe-auth/AMwmfiYbYYtKvTbJSmmuDqYetLa2mVRFks5tQxLfgaJpZM4R1QPS.