Offline Surface Tracker and Pupil Player crash issue

[darksipu]

Hi there guys,

So, we have 2 issues:

First, we ran Pupil Capture and recorded files around 1 minute with the source code version on a mid 2010 iMac. When we try to open any recordings made by using Pupil Capture on any other Macbook Pro running the GUI version of the Player, it instantly crashes. We tried with different file lengths and it doesn't work. Would it be a problem of using source code vs GUIs?

Second, the Offline Surface Tracker crashes when we want to export data with it in the Player (running the source code). In some instances, it also just crashes when you try to edit the surface that has to be tracked.

If you can give us a hand, that'd be brilliant as we have to analyse some data for a research project that has to end relatively soon. Our aim is actually to get a surface defined on a screen and get the raw eye-gazes in this region, eventually splitting it up into pixels/different regions.

Thanks, Florin

[papr]

It would be very helpful if you could upload the output of your terminal (assuming that you are running from source) to http://gist.github.com and paste the link below.

[darksipu]

https://gist.github.com/darksipu/1f144f11b210c842992c7caaedd30772 This is opening a video, opening offline surface tracker, then adding a surface and it stopped working.

https://gist.github.com/darksipu/5e4121d1a0adc92c64efc10989edf421 This one is similar, as I tried to edit a surface.

For another video, I did manage to export from the offline surface tracker what was needed. Still, I have to go through 80 videos and get a surface tracked for each.

Thanks for the help.

[darksipu]

And I tested a video recorded with the GUI Capture on the MBP on the source-code-run Player on the iMac and it worked. The issue only goes the other way around.

[papr]

Hey @darksipu, thank you for the crash logs.

Could you clarify under which condition it does not work, i.e. Capture version + OS + Hardware during recording and Player version + OS + Hardware of the system on which you try to open the recording.

Am I correct to understand that only old recordings make Player crash?

[darksipu]

So, recordings were made with a source-code-run 0.8.7.w.67 Capture on a El Capitan 10.11.6 iMac (2.8GHz, mid 2007, 4GB RAM, DDR2). On the same machine, the Player (same version) gave the crash reports that I've sent you.

I tried to open the recordings on a GUI 0.8.7.1 Player running on a Sierra 10.12.2 MacBook Pro (2,4GHz i5, 8GB RAM, DDR3). It crashed instantly.

And any recording made on the source-code Capture crashes the GUI Player.

[papr]

The 0.8.7.1 GUI Bundle runs Python 2.7 while the 0.8.7.w67 software runs Python 3. A new bundle with Python 3 for macOS and Linux will be released soon.

In general, recordings are not guaranteed to be backwards compatible. The crash on the iMac will be investigated though.

[darksipu]

I see. When do you reckon that would be? I might just install the source-code on my MBP to get it done then.

[papr]

Today or tomorrow. Setting up the source code version is a good idea though if you have the knowledge to do so. This way you will receive updates faster since you can simply pull from master.

[papr]

Should be fixed with 9d5ec1aa6ae413f96b2f6996d9742cec4130c07f in #601

[ogik13]

Hi darksipu,

Has your issue with Player crashing been resolved and how? I am having the same issue but I am not a Mac user and don't run it from source. I couldn't figure out the solution from the git post #598. Any help is appreciated!

Ogi

[darksipu]

Hi there,

So, I'm not 100% sure how to deal with it if you don't run if from source. What I did was to change some lines in the source code, according to 9d5ec1a mentioned by papr above and then it worked.

I guess asking these guys about how to deal with the GUI version is the best thing. I do find it easier to run it from source code though - it's very useful to deal with errors like these.

Hope that helps

[ogik13]

Hi,

Thank you for the reply. It seems like running it from the source would be the only option until a new bundle with a fix comes along. I will try that.

Thanks.

On Tue, Feb 14, 2017 at 8:01 AM, darksipu notifications@github.com wrote:

Hi there,

So, I'm not 100% sure how to deal with it if you don't run if from source. What I did was to change some lines in the source code, according to 9d5ec1a https://github.com/pupil-labs/pupil/commit/9d5ec1aa6ae413f96b2f6996d9742cec4130c07f mentioned by papr above and then it worked.

I guess asking these guys about how to deal with the GUI version is the best thing. I do find it easier to run it from source code though - it's very useful to deal with errors like these.

Hope that helps

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pupil-labs/pupil/issues/598#issuecomment-279730724, or mute the thread https://github.com/notifications/unsubscribe-auth/AXRM6gR6pjdMpm8bXy2F53ALVXmtuWYeks5rccGygaJpZM4LsoXs .

[Aeefire]

Would it be possible to get a windows-bundle with this fix included in the next few days, in case the release is not coming anytime soon?

[willpatera]

@Aeefire - working on the Windows bundle as we speak.

[willpatera]

@Aeefire - Pupil Capture is working on Windows now - but a few more minor changes and more testing is required before releasing but should be online by the end of the day today. We will also hope to add Linux bundles later today as well.

[Aeefire]

so i tried the new version; while surface tracking now worked longer, the same error still appears:

[willpatera]

@Aeefire - Thanks for the report. I'd just like to double check - you are running Windows 10 correct? This screenshot is from the cmd prompt when running the bundle or from source?

[Aeefire]

@willpatera bundled, Windows 10; I have not been able to reproduce why this happens. I had the eye tracker worn now for over 10 minutes, having the surface go in and outside of the FOV all the time, without crashing again.

[willpatera]

@Aeefire - While I have never been able to reproduce this issue myself, your report is helpful.

Based on the error message, I suspect that one of the arguments to cv2.perspectiveTransform() is sometimes an incorrect type or None.

[Aeefire]

I've played around with it again and saw that the crash only happens when an eye is active and suspect that it only happens when you gaze into the surface area.

Apparently this fix https://github.com/papr/pupil/commit/9d5ec1aa6ae413f96b2f6996d9742cec4130c07f hasn't made it into the master branch / v0.9.0 release, why is that? Seems like the issue is fixed already.

edit: https://github.com/pupil-labs/pupil/issues/631 here it's stated that it will be in the next release (this message was before 0.9.0) but apparently hasn't found its way into the build, it's also not in the current master branch as it's been reverted here: https://github.com/pupil-labs/pupil/commit/e02d217a01ace0f68a743161daf83f06f7c3bf8d

ping @willpatera @papr

[papr]

@Aeefire The reason we did not include it, was that this is a "lazy fix" that does not really fix the underlying issue. We will have to investigate the underlying issue.

[Aeefire]

Thanks. It was a bit confusing to not see it working as your comment on the other issue stated that it's fixed and going to be part of the next release.

As I had a hard time (i.e. could not get it running from source), still my question: Is it possible to get a custom windows package with this fix included?

Otherwise thanks for your tremendously fast responses and the good work!

[papr]

Dear @Aeefire unfortunately this is not possible. We are aiming on releasing a correct fix with the 0.9.2 release. Until then, your only other possibility is to follow my advice of cherrypicking the lazy fix.

We will be releasing a new driver installer for Windows very soon. This will hopefully improve the Windows source installation process.

[florianbussmann]

Hello @papr, as I am struggling with the same issues regarding cv2.perspectiveTransform() I applied your fix from the mentioned commit. However this leads to another error which I could not fix by mself

Traceback (most recent call last):
 File "C:\dev\tools\Python\Python35\lib\multiprocessing\process.py", line 249, in _bootstrap
   self.run()
 File "C:\dev\tools\Python\Python35\lib\multiprocessing\process.py", line 93, in run
   self._target(*self._args, **self._kwargs)
 File "J:\source\eyetracking\dev\trunk\pupil\pupil_src\capture\world.py", line 453, in world
   p.recent_events(events)
 File "J:\source\eyetracking\dev\trunk\pupil\pupil_src\shared_modules\plugin.py", line 66, in recent_events
   self.update(events['frame'], events)
 File "J:\source\eyetracking\dev\trunk\pupil\pupil_src\shared_modules\surface_tracker.py", line 208, in update
   s.gaze_on_srf = s.map_data_to_surface(events.get('gaze_positions',[]),s.m_from_screen)
 File "J:\source\eyetracking\dev\trunk\pupil\pupil_src\shared_modules\reference_surface.py", line 428, in map_data_to_surface
   return [self.map_datum_to_surface(d,m_from_screen) for d in data]
 File "J:\source\eyetracking\dev\trunk\pupil\pupil_src\shared_modules\reference_surface.py", line 428, in <listcomp>
   return [self.map_datum_to_surface(d,m_from_screen) for d in data]
 File "J:\source\eyetracking\dev\trunk\pupil\pupil_src\shared_modules\reference_surface.py", line 422, in map_datum_to_surface
   mapped_pos = typesafe_perspectiveTransform(pos , m_from_screen )
 File "J:\source\eyetracking\dev\trunk\pupil\pupil_src\shared_modules\reference_surface.py", line 44, in typesafe_perspectiveTransform
   return cv2.perspectiveTransform(A.astype(np.float32, copy=False), B)
v2.error: D:\Build\OpenCV\opencv-3.1.0\modules\core\src\matmul.cpp:2125: error: (-215) scn + 1 == m.cols in function cv::perspectiveTransform

norm_pos is (6.6282682703837636, 9.0184701197332942)
pos is [[[ 6.62826827  9.01847012]]]
m_from_screen is [[[ 0.          0.21078603  1.21078598]
  [ 0.          0.05219692  1.05219698]]

 [[ 0.          0.86407661  1.86407661]
  [ 0.         -0.04093897  0.95906103]]

 [[ 0.          0.89153159  1.89153159]
  [ 0.          0.64041108  1.64041114]]

 [[ 0.          0.22406058  1.22406054]
  [ 0.          0.70952964  1.70952964]]]

[mkassner]

For this we need to figure out what the data is that causes the exception. Could you add a print before the offending line so we know what the wrong input is and trace that back? We need to understand the type and content.

[mkassner]

Could you send us a recording that creates this error? Then we can reproduce and fix.

[florianbussmann]

The values attached above are causing the exception

pos = np.array([d['norm_pos']]).reshape(1,1,2)
mapped_pos = typesafe_perspectiveTransform(pos , m_from_screen )

Actually this is not a problem with the Player app for me, it occured in Pupil Capture once the gaze is on the surface

[mkassner]

Ah ok. That should be reproducible. I ll have a look once I have my hands on an eye-tracker.

[mkassner]

I tried recreating this error in realtime using surface tracker with a few different surfaces and all combinations of 2/3D bino/monocular but could bot trigger the error. Does anybody have a recording that is known the reproduce the error?

[Aeefire]

I'll try to provide you with one next week, when I am back at work!

Side note : I always manually edited a surface with 2 markers to a bigger surface than the one delimited by the markers.

[Aeefire]

Hey,

So I had the chance to do another test. The thing is, I am not quite sure on how to provide you with a recording when the application crashes and doesn't correctly create the recording.

I could still reproduce this error on the newest version (0.9.1) with the new windows drivers. Setups I tried (which ALL lead to a crash) were: binocular, monocular + 1 surface with 4 markers, 1 surface with 2 markers and manually extended area.

All setups lead to a crash after "looking into" the area,

Examplary screenshot:

and a link to a fraps-video of it:

https://www.dropbox.com/s/02aq5nrtzgcj1iz/pupil_capture_surfacecrash.avi?dl=0

[papr]

Dear @Aeefire @cpicanco @florianbussmann Since we are not able to reproduce this issue on any of our machines (this includes all supported OS) we need you to checkout papr/pupil:fix598, run Pupil Capture such that it crashes, and post your log messages below.

This branch includes additional log messages that will hopefully help us to fix this problem.

[Aeefire]

Well, I'd love to but I am stuck at following message when trying to build:

"Unbekannte Option "...." wird ignoriert" means "Unknown option ... is ignored"

[papr]

@Aeefire Please refer to #612 for windows install related discussions to keep this issue on topic.

[Aeefire]

@papr So I somehow managed to get it running with your patch and am getting now the following debug messages:

[Aeefire]

Hey @papr,

I think I found the issue.

In shared_modules\calibration_routines\gaze_mappers.py#_clamp_norm_point(pos) you're

return min(100,max(-100,pos[0])),min(100,max(-100,pos[1])) clamping float values using integers as min/max values.

According to https://docs.python.org/2/library/functions.html#min this will select the smallest / biggest item of an iterable. I.e. when a value smaller than 100 is reached, it will SELECT the integer 100 instead of a float. Changing the min/max values to 100.0 (i.e. a float) seems to have fixed the issue for me (without using the typesafe_perspectiveTransform helper method)

[papr]

Wow, @Aeefire, very good find! We will incorporate your fix into the master branch as soon as possible.

[mkassner]

return min(100.,max(-100.,pos[0])),min(100.,max(-100.,pos[1]))

is all that was needed :-)