Any way of measuring blink amplitude?

[MFernandezAlonso]

Hi,

I was wondering if the Pupil labs software has any way of extracting a measure of blink amplitude from the eye videos? Or at least whether a blink was complete vs incomplete?

Since there is a 3D model for pupil detection, I was wondering if this could be used to detect how much of the eye is covered by the eyelid at each blink event.

Thank you.

[papr]

Dear @maydel

The 3d eye model only models the eye ball and the pupil. It does not model the eye lid. Therefore, one can not use it to calculate blink amplitudes. You can read more about it here: https://www.researchgate.net/profile/Lech_Swirski/publication/264658852_A_fully-automatic_temporal_approach_to_single_camera_glint-free_3D_eye_model_fitting/links/53ea3dbf0cf28f342f418dfe/A-fully-automatic-temporal-approach-to-single-camera-glint-free-3D-eye-model-fitting.pdf

Pupil Core detects blinks implicitly by testing the pupil data confidence stream for sharp, binocular drops. This has proven to be an effective and fast method for blink detection. Unfortunately, it does not provide the possibility to calculate the blink amplitude either.

Detecting the eye lid is a hard computer vision problem and currently not in the scope of this project.

[peteratBHVI]

Hi @maydel,

what do you need if for? I was working on a method to measure it. cheers Peter

[papr]

@peteratBHVI Cool! Have you published this method yet? Are you able to share your approach?

[MFernandezAlonso]

Hi @peteratBHVI,

Thanks for your reply. It's for academic research, for analysing data from a study I did on visual fatigue. I found an effect on observer's blink rate, but it would be great to have a way to evaluate blink amplitude as well, since from observation of the recordings, it seems to be quite a relevant measure. It would also (potentially) solve the issue of some incomplete blinks not being detected by the existing blink detection algorithm (for which decreasing the threshold hasn't worked either, as it just creates lots of false positives).

I would appreciate any advice/insight into what is the best way to approach this. I have been looking into some approaches (e.g. http://vision.fe.uni-lj.si/cvww2016/proceedings/papers/05.pdf), but so far have only found them available for full-face recordings, rather than eye-only recordings.

Thanks in advance.

[peteratBHVI]

@papr there are a couple of papers out with face on cameras (Ioana Bacivarov), not so helpful. but have a look to this presentation. "The influence of methodology on blink dynamics" Maria Navascues-Cornago, Michael L Read, Carole Maldonado-Codina and Philip B Morgan

basically I try the same.

• selecting all images belonging to a blink
• image histequalisation to have more similar eye images
• slice them vertically around the pupil center if consecutive good confidence
• find the interval of interest in the slice and find the edge / threshold

accuracy is hard to determine, but a completeness of the blink can be determined.
white - found edges red - average of 10 first and last with high confidence green - filtered lid edge while low pupil confidence

there are still a lot of inaccuracies, but overall not so bad. my filter takes advantage of the fact that the eye lid reflects ir light well and is very close to the camera, hence it is very bright in the image. works better in the top and middle of the eye image than at the bottom.

[peteratBHVI]

[peteratBHVI]

AAO 2019 poster blink dynamics-min.pdf

[peteratBHVI]

10-20% of images are like this, the rest is actually better.

I'm looking for a way to run the PL blink detector automatically with different onset / offset confidence thresholds. Looking then at output values like duration allows to optimize onset / offset. That could be positive for further image analysis.

cheers

[peteratBHVI]

AAO 2019 poster blink dynamics-min.pdf

can you download this? cheers Peter

[PWaRoo]

As a start, if the eye video quality is ok, it should work out the amplitude. https://github.com/PWaRoo/Blink_amplitude_analysis

still need some improvements and maybe some customization for individual eye colors. Happy about any comments. Cheers Peter

[papr]

@PWaRoo Cool! Feel free to make a pull request to https://github.com/pupil-labs/pupil-community adding your project to the "Scripts" category.

[PWaRoo]

@papr I will do. Just going through a bigger data set and refine a few things. I hope within the next couple of weeks. cheers