Profile current resource utilization and improve performance

misterpeddy / hands-down

Slow the spread of COVID-19 by discouraging people from touching their face

6 stars 1 forks source link

Profile current resource utilization and improve performance #7

Open misterpeddy opened 4 years ago

misterpeddy commented 4 years ago

Currently the performance of the application is borderline -- if we do a bit more work per inference, we'll definitely start to slow down some more limited devices.

First we have to profile to understand where the cycles are going. How much compute resources is each of handpose, facemesh, inference, drawing utilities, and other utilities consuming?

To make things slightly more complicated, we have a few backends that all the models could be executing on: CPU, WASM, WebGL - Read more about it here

Ideas on how to improve performance:

instead of running per frame redraw (here) run at fixed interval (once a second?)

amirhmk commented 4 years ago

Another idea on improving performance could be introducing a threshold on displacements from the face before running the inference model?

WebGL looks pretty cool and it seems like it can buy us time, would be good to look at

amirhmk commented 4 years ago

I think the idea of frame-by-frame could be good, it may be better suited if we were to predict a user is about to touch their face by tracking the route?

Berkmann18 commented 4 years ago

I think the idea of frame-by-frame could be good, it may be better suited if we were to predict a user is about to touch their face by tracking the route?

Yeah, using a bit of physics to predict that could be a good idea, but I can see some (potential) caveats to that:

will it be more efficient and faster than doing the frame-by-frame way?
will it be accurate and account for edge cases where the hand would change direction right before touching the face?
how would the two above play out if the direction/speed/... of the face's movements were taken into account?