search

esdalmaijer / PyGaze

an open-source, cross-platform toolbox for minimal-effort programming of eye tracking experiments
www.pygaze.org
GNU General Public License v3.0
671 stars 211 forks source link

Tobii Pro SDK support #98

Closed alisdt closed 6 years ago

alisdt commented 6 years ago

PyGaze doesn't currently implement the new Tobii SDK, module tobii_research:

http://developer.tobiipro.com/python/python-sdk-reference-guide.html

I've successfully adapted an experiment to use this SDK using inline code in OpenSesame, so I think I'm in a position to try and add this. I'm creating this issue to flag it up to anyone else who is interested (and as a reminder to myself!).

It seems that older Tobii devices don't support this new SDK, and newer devices don't support the old one -- and I'm not sure whether there are any in the middle that support both.

This SDK uses a callback which can cause problems in OpenSesame, see:

https://github.com/smathot/OpenSesame/issues/545

(the solution is probably to restrict what's done in the callback, rather than making changes to OpenSesame)

arnohakk commented 6 years ago

We are using the TX300 (which to my knowledge supports both SDKs) and would also be interested in migrating, perhaps it would be good to stay in touch.

grebdems commented 6 years ago

Hi @alisdt ! Magnus from Tobii here. Funny that you would happen to mention this now, as we (at Tobii Pro) are just looking into doing exactly the same! Maybe we can bounce some ideas on how to best do this? I'm guessing you have more experience with PyGaze, and we have more experience with the Tobii Pro SDK...

Some questions we're considering:

  1. Should this be a new trackertype? (or should it replace the exising one?) I'm leaning towards adding it as a new type, as the old class has a lot of methods that are not in BaseEyeTracker, and there's no knowing if someone is using them or not.
  2. Should the new type retain the output/log format of the old one? It's somewhat limited (doesn't include pupil data, for example), but again, I don't know if there are people out there that have built analysis code on this format (noticed that there is no support for this format in PyGazeAnalyzer though).

Regarding device support, Tobii Pro SDK supports almost all of Tobii's older eye trackers (see http://developer.tobiipro.com/tobiiprosdk/supportedeyetrackers.html for full list). Essentially, it should support any model that the previous SDK did, but we have "only" verified it with the ones on that list.

Btw, I'm leaving for vacation now, so I will probably not personally push any code here, but my colleague should probably be able to get something up during the coming weeks!

alisdt commented 6 years ago

@arnohakk Interesting, I ended up using the new SDK because our TX300 refused to work with the old plugins.

alisdt commented 6 years ago

@grebdems

Thanks, it's great to see Tobii taking an interest in free software!

  1. I'm not that knowledgeable in PyGaze but I would imagine a new type would be advisable. This would allow people to keep running old experiments on older equipment without having to keep a previous version of PyGaze around.

  2. My code just logged everything (including OpenSesame loop variables, which caused the bug I referenced above). I'll upload an example experiment soon and make it license compatible with PyGaze / OpenSesame, in case you want to incorporate part of the code into a PyGaze PR. For us, I know that pupillometry would be important.

esdalmaijer commented 6 years ago

Great to see so many people are looking to help PyGaze advance! As for the questions above:

  1. I definitely think it would be best to write a completely new class, e.g. TobiiNewSDK or something. In fact, as Tobii itself suggests that the new SDK should work with older models too, I would be in favour of renaming my older Tobii class, and making the new SDK class the default. The new SDK should inherit the BaseEyeTracker class, and most of it's functionality should be implemented.

  2. It's not necessary, and I'd be happy to write an extension for PyGaze Analyser to read the new format (once I actually have an example, as I currently don't have access to a Tobii machine).

grebdems commented 6 years ago

Wow! Great with a lot of action on this one! @esdalmaijer We'll go with your suggestions. That gives us the most freedom! Do you want us to push some intermediary commits on a branch, or should we just push everything as a pull request once we think we're done? Do you have any more examples or tests that we can use to verify that everything works ok? (except for the ones in examples/)

arnohakk commented 6 years ago

@alisdt We are using Pygaze with the old SDK with the TX300 for two years now, it works fine. @grebdems Great to hear that Tobii is active here as well, it would be great to have an online fixation algorithm in either the SDK or Pygaze since all of the code we found so far does not consider the freely moving head. The Pygaze algorithms are still somewhat working fine, but are "wrong".

alisdt commented 6 years ago

As promised here's my code:

https://github.com/alisdt/tobii-pro-sdk-opensesame-example

Once support has been added to PyGaze I'll go back to make it clear that it's obsolete. For now I've left a warning for folks to check PyGaze / OpenSesame support before using my code.

esdalmaijer commented 6 years ago

Perhaps of use: @oguayasa has been developing a Python wrapper for the new Tobii Pro SDK too. https://github.com/oguayasa/tobii_pro_wrapper

@grebdems: Please use whatever way of working you like best. I'd be happy to provide comments on step-by-step commits, but would also be happy with you doing a pull request once everything is finished. (Please do keep in mind that I don't actually have access to a Tobii tracker, so I rely on people in this thread and others to do the actual testing.)

As for the testing: The examples are a bit dated, I'll see if I can set up some newer ones :)

oguayasa commented 6 years ago

thanks for posting @esdalmaijer ! If anyone has a tobii tracker and runs into bugs with the wrapper, please let me know.

As a quick side note, Tobii has also put out an "upgrade key" that should make older model non-Pro trackers compatible with the new SDK as well.

grebdems commented 6 years ago

Pull request: #100

esdalmaijer commented 6 years ago

Thanks! Very useful addition! As I mentioned before, I'm not in a position to test it (no Tobii trackers around), but I will advertise this in the hope that people will give your code a test run on their own systems. (I assume you already thoroughly tested this, but still good to advertise the new option.)

I've also added OpenSesame support by updating the plug-ins. See https://github.com/esdalmaijer/PyGaze/commit/09f183663789b89adb7517498b2f8daa2c042f1b

alisdt commented 6 years ago

I'd like to add my thanks! I'll test this as soon as it makes its way into an OpenSesame release.

esdalmaijer commented 6 years ago

Pinging @smathot. You back from all your conferencing yet? ;)

You might be interested in the above, #100, and https://github.com/esdalmaijer/PyGaze/commit/09f183663789b89adb7517498b2f8daa2c042f1b

grebdems commented 6 years ago

Thanks! We're happy for all real usage tests! As mentioned before, we're far from experts in PyGaze, so please let us (me or @pedrotari7) know if there are any issues!

esdalmaijer commented 6 years ago

Thanks, @grebdems! Looking through your and @pedrotari7's code, I got the idea that things were implemented well.

One question that came up on Twitter earlier, was whether the EyeX and C4 are supported through the new SDK. The website seemed to suggest they weren't, but thought I should ask anyway. Are they?

oguayasa commented 6 years ago

No, I don't believe so. It's just the pro line of eyetrackers and their new vr headset.

On Sep 7, 2017 5:52 PM, "Edwin Dalmaijer" notifications@github.com wrote:

Thanks, @grebdems https://github.com/grebdems! Looking through your code, I got the idea that things were implemented well.

One question that came up on Twitter earlier, was whether the EyeX and C4 are supported through the new SDK. The website seemed to suggest they weren't, but thought I should ask anyway. Are they?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/esdalmaijer/PyGaze/issues/98#issuecomment-327842411, or mute the thread https://github.com/notifications/unsubscribe-auth/AbNdFhuLiTfwgZ5uu9zPdgM91W48hGc-ks5sgBE6gaJpZM4OeBWd .

grebdems commented 6 years ago

All credit for the code should go to @pedrotari7! My job is more pointy-hair-boss:y, point at the screen and say "write more code there", that sort of thing... :-P

The 4C is supported, but requires a license. I know it's a bit ambiguous from the docs right now, but that's sort of on purpose. For now we're handling interest on a case-by-case basis. If you're interested in using the 4C for research, go to https://www.tobiipro.com/contact/contact-eyex-for-research/

scattenlaeufer commented 6 years ago

If I read the code correctly, there is at the moment no way to get external trigger signals from the data provided in PyGaze. To be honest there also isn't a really good way to get this data in tobii_legacy, but since we needed it and thanks to the way data is provided in the old SDK, we were able to botch ourselfs a way to get the data. But in the new SDK, external trigger signals aren't part of the eyetracking data anymore, but are provided by tobii_research.ExternalSignalData instead. Is there anyone, who needs this is maybe already is working on a way to use it? If not, I'd think about a way to make it usable and try to add it to PyGaze.

esdalmaijer commented 6 years ago

Thanks for clarifying, @grebdems!

@scattenlaeufer: I haven't heard from anyone who's done that yet, so would be a great addition if you could find a way to integrate it in PyGaze. Maybe the Tobii developers would be happy to help if you have any questions?

bruno2git commented 6 years ago

Hi everybody! (Pinging Tobii developers @grebdems and @pedrotari7)

I've tried both the Old and the New Tobii SDKs in PyGaze and OpenSesame, with a Tobii Pro X2-60 eye-tracker (in Windows 10).

While the experiment seemed to run well with both, I have not been able to align the eye-tracking data with the experimental events using the New Tobii SDK. I have created eye-tracker log events immediately before and after the stimulus display and, while the difference between the time-stamps of these events matches the display time of the stimulus (~3000 ms), the slice of data that fits in-between these time-stamps misses more than one second of data.

TobiiTest.zip contains a very simple experiment in OpenSesame, the log files generated using the Old and the New Tobii SDKs and excel files evidencing what I explain here. If necessary I can provide an example in PyGaze at a later time, since I don't have access to the eye-tracker presently, but the results in PyGaze were identical. I've also tried start/stop recording on a trial basis versus the whole experiment but I wasn't able to align the events with the data in either case.

Other issues are listed in the following forum conversation.

Thank you all for your work and cheers! :)

esdalmaijer commented 6 years ago

The Tobii Pro implementation seems to run smoothly, so I'm closing this thread. If any problems occur, people should absolutely feel free to open a new and specific issue.

Thanks again @grebdems and @pedrotari7 for implementing this!

nym commented 5 years ago

@grebdems and @pedrotari7 is it true it costs a minimum of $2,000USD to get a 4C upgrade key? I just want to do AT R&D for OSS.

grebdems commented 5 years ago

@nym I don't have the exact numbers (not my domain), but it's somewhere in that ballpark. I don't think we have any OSS discounts unfortunately (again, not my domain). Check with sales@tobii.com if you're interested!

nym commented 5 years ago

@grebdems that's who told me $2,000 unfortunately. The use case did not affect the price according to them.