Inaccurate Iris/Pupil Localization

[Spatiumensura]

We have recognized an inaccuracy of the pupil localization, which seems to occur at the right eye only (from camera perspective)

• cropped part of the input image (1280 pix x 720 pix, fx: 6308.377 fy: 6279.325 cx: 541 cy: 403)
• cropped part of the result image (the right iris is misaligned and does not adapt to the actual size of the iris and pupil)

The misalignment of the right iris seems to be systematic, because it occurs for all users in all our recordings. We have observed this phenomenon also with a different camera (web camera). We think that the image quality should allow for a better result. Can we overcome this by adjusting the parameters of the landmark localization (which parameters)?

Many Thanks!

[TadasBaltrusaitis]

Interesting, I have not observed this behavior before. The left and right eye landmark detectors are in fact mirrored versions of each other, so if one has a bias in a misalignment so should the other. If there is a systematic bias in the illumination in your data that might explain why this would be happening.

I will have a look at this though on some of the evaluation datasets to see if there is a bias in error.

[Spatiumensura]

We have flipped some of our images and applied the landmark detector:

Each row shows the original and the flipped image and the detection results. According to your statement, that the landmark detectors are mirrored versions of each other, there should be a mirrored result of the pupil detection. This seems to be not the case (especially for the first and last row). In the second row, the results seem to be consistent, but the estimated size of the right pupil (from camera perspective) in the flipped image is larger compared to the left eye in the original image.

[TadasBaltrusaitis]

While models for the eye landmarks (patch experts and point distribution models) are flipped versions of each other, they are not flipped for the entire face landmark detection which is used to initialize the eye landmarks. Also flipping the image will cause face detection results to change a bit as well, leading to a not identical resulting stages. This would cause some differences in the results.

But that may point where the bias could be coming from.

[Junxen]

I encounter the same problem on cell phone camera, any idea to fix it?

[TadasBaltrusaitis]

What exact problem are you experiencing? I'm still trying to figure out if it's a bug or not.

[mitchelkappen]

Hi Tadas,

I encounter the same problem as Spatiumensure mentioned. It only occurs at the right eye and I cannot seem to figure out a certain pattern. It occurs over different datasets, recorded in different settings. It is not entirely consistent, but happens pretty frequently.

[TadasBaltrusaitis]

Can you share some example images of where this happens for me to have a look at?

[sarratouil]

@TadasBaltrusaitis i want to detect the iris in real time any solution please?

[TadasBaltrusaitis]

If you have access to full face images/videos you can already do that using OpenFace. It contains eye landmarks corresponding to the iris. See https://github.com/TadasBaltrusaitis/OpenFace/wiki/Output-Format for more details.