jeffhuang
commented
3 years ago a quick hack could be to just randomly sample a few pixels to check if they've changed. it's unlikely that a couple of pixels would not change their exact color at the same time
Currently, WebGazer runs as fast as it can on whatever image is in the video element canvas. Ideally, it would only run when a new frame is added, to reduce redundant work.
This can be tracked via non-standard-HTML ways, e.g., by looking at when the
One caveat of this simple implementation change is that clmTracker is actually iterative in its convergence: even if the video frame hasn't changed, clmTracker will still do useful work converging to the true face when run again. It is again a simple change in the code to still run clmTracker even if the frame hasn't changed. Ideally, WebGazer would run clmTracker as often as it can, but only run eye gaze prediction when a new frame is decoded.
As such, the gain is thus: you can save the work that WebGazer does in making predictions on duplicate frames, and give it to clmTracker to get better face tracking estimates. On my laptop (Core i5 from 3 years ago), the gain is pretty minimal because running WebGazer at 60Hz is already using most of a core, but on a desktop this gain could be better.
In general, it's better if WebGazer does only the work that it needs.