Tobias-Fischer / rt_gene

RT-GENE: Real-Time Eye Gaze and Blink Estimation in Natural Environments
http://www.imperial.ac.uk/personal-robotics
Other
365 stars 68 forks source link

Make estimate_gaze_standalone.py work without cuda #68

Closed alsuren closed 4 years ago

alsuren commented 4 years ago

My home dev machine is a 5 year old macbook, and I was curious to try out your project (which I found on paperswithcode). After a little bit of patching, I managed to get it to work without cuda, and it works really well.

It's a bit too slow to process 1080p video frames (unsurprisingly), but if I restrict myself to only looking at keyframes (~every 2-4 seconds), and I extract face chips using opencv+dlib (which I'm already doing for another part of my project) then it is fast enough to iterate on.

This is the patch I needed to make it work without cuda.

Now that I know that I have the standalone example working, I will try integrating it into my project, and see if anything else needs tweaking.

Tobias-Fischer commented 4 years ago

Hi @alsuren, Many thanks for your contribution and interest in our work! I will let @ngageorange deal with the PyTorch stuff, but I just wanted to mention that you'd be welcome to create another PR with a script that extracts the face chips using dlib if you want to.

Let us know if everything else goes okay.

Thanks again, Tobias

alsuren commented 4 years ago

@Tobias-Fischer my current face detection is based on https://www.learnopencv.com/face-detection-opencv-dlib-and-deep-learning-c-python/ (https://github.com/spmallick/learnopencv/blob/master/FaceDetectionComparison/face_detection_opencv_dnn.py) with the dlib frontal face detector to narrow things down. It's a bit of a scatter-gun approach, and it misses a lot of faces, so it probably doesn't belong in this repo.

Tobias-Fischer commented 4 years ago

Hi @alsuren, That makes sense, no worries. Thanks again for your contribution. We are currently preparing a release and will merge this PR soon (after @ngageorange confirms that your latest commit is what he had in mind).

Best, Tobias