Trigger sending on two PC Setup

[paul-gallenkemper]

Hello again, In this next step of our Setup, we want to send triggers to Tobii Pro Lab via talkToProLabs "sendCustomEvent" function. To do so, we currently use: e.g. talkToProLab.sendCustomEvent(TobiiProSDK.get_system_time_stamp(), sprintf('Block number %d End of Block', block));

meaning we chose Tobii Pro SDKs System Timestamp as the trigger timestamp. This mostly works, but this error is thrown, when the two computers were turned on at two different timepoints. It rarely occurs, when the computers were turned on around the same time (seconds):

"TalkToProLab: Command 'FinalizeRecording' returned an error: 201 (Recording finalization failed): Custom event timestamp is out of recording time bounds"

What lead us to chose this timepoint are the TalkToProLab documentation and the documentation here: https://connect.tobii.com/s/article/Why-do-I-have-inconsistent-timestamps-in-the-data-export-file?language=en_US

In TalkToProLabs documentation, it is stated that "you are responsable to present talkToProLab with timepoints in Pro Lab time, not local time". In Tobiis documentation, it is mentioned that the System Timepoint, depicted as the Computer.timepoint in output tables, is used for Tobii Pro Labs internal timing and is also provided by Tobii Pro SDK (i assume via get_system_time_stamp). I have no idea why this error seems to be thrown at all, because this timepoint seems to me to be the correct one. I also don't know why this error seems to the thrown depending on the synchronization status of the two computer's clocks, when neither of the two functions (sendCustomEvent or get_system_time_stamp) use local time.

I looked a little bit deeper into possibilities of getting the Tobii Pro Lab time directly into Matlab on the operating PC. It seems that you would either have to build another Websocket Server to tap into the win32 timestamp on the Pro Lap PC or you would have to modify the Websocket client provided by Titta. Neither seems like something i would be comfortable doing.

Do you have an idea why the Tobii Pro SDK System timestamp might be wrong and what would be a suitable alternative to feed into the sendCustomEvent function?

Best Regards, Paul

I'll add my colleague as well, she has been in contact with you over this topic: @ninaalisahinz

[dcnieho]

As said in #102 (please read through all of that), you need the system time stamp on the Tobii pro lab computer, not on the stimulus computer. They happen to be similar enough when you switch on the computers at around the same time, but that is not a solution. They'll be off, just not as far that you get that error, but nonetheless data and stimuli/custom events will not line up.

Neither of you suggestions is what is needed. All that is needed is to determine the offset in clocks between the two computers, and to apply that offset. That requires some new code to be implemented, using the clientClock object in TalkToProLab. Do you have a timeline for when you need this to work? Then i can indicate if it is realistic that i may have a chance to look at it.

Also, could you tell me exactly how the two computers and the eye tracker are connected to each other? What is your network setup? What IP do you use to connect to the eye tracker on the two computers? Just so i don't have to figure that out when i work on this, saves time.

[paul-gallenkemper]

I am familiar with Issue #102, Nina and I have been working on that together.

I am confused why you think the system timestamp collected by get_system_time_stamp using Tobii Pro SDK (which is btw the same as the System.timestamp saved in the gaze data buffer, which can be viewed by peekN('gaze')) is local to the Matlab PC and not the Tobii Pro Lab PC. All of Tobiis documentations (see message above, but also here: https://developer.tobiipro.com/commonconcepts/timestamp-and-timing.html "system time stamp uses the computer on which the application is running as the source") lead me to think that the timestamp gathered this way is actually local to the Tobii Pro Lab PC and should therefore be compatible with Pro Lab.

Let's still continue with the assumption that this system timestamp is local to the Matlab PC.

Regarding our schedule: We originally wanted to start data collection long ago, but now we actually wanted to invite some pilot participants for next Monday as everything apart from the trigger sending and some inconsistencies with PTB is now ready. I don't want to decide anything final without my colleagues and my boss, but let's say this should be ready by the end of next week. If this is not convenient for you, no problem at all! You have been of great help already!

This is our network setup: The eyetracker and both PCs are connected to a LAN switch. Nothing else is on the LAN switch, but both PCs are also connected to the universities LAN, which is also connected to the internet. The Matlab PC (Linux) has the IP address 134.100.144.154 in the local LAN. The Tobii Pro Lab PC (Windows) has the IP address 134.100.144.165 The eyetracker has the IP address 169.254.11.151, either given as tet-tcp://169.254.11.151 (on Windows Eyetracker Manager) or as tobii-prp://169.254.11.151 (on Linux Eyetracker Manager).

To connect to Tobii Pro Lab, we use: talkToProLab = TalkToProLab('Project29', true, '134.100.144.165'); (sidenote: I've tried this with DoSynchCheck set to false, doesn't really seem to change anything systematically).

To connect to the ET directly, we use: EThndl = Titta(settings); EThndl.init('tet-tcp://169.254.11.151') with the settings predefined for advanced image calibration.

Additionally, we simply use TobiiProSDK = EyeTrackingOperations(); to set up a Tobii Pro SDK instance that can get the system timestamp

[dcnieho]

This friday is something i certainly can't promise.

Hmm, so 169.254.11.151 is reachable through the switch, despite being in an entirely different subnet than 134.100.144.154? I worried about that working... Good to hear that it does.

"system time stamp uses the computer on which the application is running as the source" -> "the application" here would be Titta, which is running on the matlab pc. System timestamps are thus on the matlab pc, not the pro lab pc.

So, to be clear, you call talkToProLab = TalkToProLab('Project29', true, '134.100.144.165'); on the MATLAB PC (134.100.144.154), and the sync check doesn't tell you that the clocks are not synced? Then my test there is broken apparently :p Switching the check off wouldn't change anything since its just a check.

You can skip TobiiProSDK = EyeTrackingOperations(); and Tobii's entire MATLAB SDK, Titta.buffer.systemTimestamp gives you the same info.

[paul-gallenkemper]

I think next friday would be OK aswell. With the first pilots, we usually only aim to find out whether the stimuli and the timing are suitable.

Yes, the connection to the eyetracker works!

I don't think the talkToProLab ever threw any errors once we gave it the correct IP address, independent of the DoSynchCheck input.

Ok, then moving forward we would have to find a way to get the Pro Lab timestamp into the Matlab PC so that we can account for the delay / offset. How would we do that considering that the talkToProLab.clientClock does not have a function that outputs the current Pro Lab timepoint?

[dcnieho]

I find it very strange that no sync errors are thrown if dosynccheck is true. clientClock, to my understanding, should be providing the Pro Lab timestamp, so what we would need to sync it all together. You know what, it is probably a bug in my check. Looking at it, it will only signal if the lab time is ahead of the matlab computer time, not when it is behind. Easily tested. This i can figure out tmr already at least.

But yeah, implementing the sync is my problem to figure out, and perhaps chat with Tobii about.

[dcnieho]

Indeed, my dosynccheck not firing was most likely a bug.

I have now fixed that bug, and also made a start at running two computer setups. Sync logic is implemented and tested when making a new TalkToProLab instance, but i am not using it for converting the timestamps provided to sendStimulusEvent and sendCustomEvent. Ran out of time for this week. Can you try out the new code and see if it crashes? As noted, this makes no observable difference for you, you will still have your original problem

[paul-gallenkemper]

I can try running the new code later today. But for now, we have decided to chose a workaround for trigger sending. We would like to record and save data either in Matlab or using Pro Lab and we'd be thankful for some advise. If we chose to record data in Matlab, we would not need Pro Lab at all, not even a second PC. To decide on this, it would be good to know whether the buffer running is a strain on working memory that might impact stimulus presentation?

If we chose to record data in Tobii Pro Lab, we would like to get rid of the buffer recording in Matlab. A suitable workaround for the timestamp synchronization would be using the device timestamp (eyetracker) as this is available to both systems. We could get this timepoint in Matlab during the stimulus presentation and use it to mark stimulus phases after recording in Pro Lab. Is it possible to get the device timestamp without the buffer running? Also, if we chose this option we would still have to open the buffer for our gaze contingent stimulus display. Does the start(‘gaze’) impact working memory or take long itself so that it might cause a delay in stimulus presentation when called during the presentation loop?

[dcnieho]

Ah, is that the only reason you want to use Pro Lab? I thought/expected it was for visualization or analysis. Eye tracking data isn't big, so there should be no issue at all. I would always recommend to record in matlab, even if you also record in Pro Lab.

This is one of these things that you should first verify there is actually a problem, before solving it. Almost completely sure there is no problem here, unless you're weirdly low on memory.

Since i have this mostly implemented now, will you see it through to the end with me, provided testing when its done? Look forward to hearing how my code runs on your machine

[paul-gallenkemper]

Ok, really glad to hear this! We actually consulted some experts from the research unit we are situated in right now as our project is a special case being the only one implementing gaze contingent stimulus display. The other projects naturally separate data recording and stimulus presentation, using various measures but also pupillometry, and advised us to do the same. But wrapping it all up in a one computer setup is a lot easier of course.

Still i'm happy to keep testing the changes you will make to TalkToProLab in parallel. As of now, nothing has changed with the new version. The talkToProLab doesn't take true / false as a second input anymore, but also doesn't throw an error when run. Matlab still connects to Pro Lab as an external presenter and finalizing our recording after sending custom events with talkToProLab.sendCustomEvent(EThndl.buffer.systemTimestamp, "trigger name") still throws the error "201: timestamp out of bounds". Looking forward to see whether the time synching will work in the future!

[iandol]

Just to say we use Titta alongside our PTB experiment manager Opticka, all running on one computer without any timing issues whatsoever, it is rock solid (though the best timing performance is found using Linux + AMD GPU). I have a Tobii Pro Lab licence, but honestly experiment design is much better handled by PTB anyway, especially if you need to guarantee precise timing or use a variety of more exotic hardware. For recording electrophysiology, we do use a second machine and use PTB as the master sync with a LabJack T4.