Unusably low framerate on camera.html

[ajbouh]

I'm seeing framerates that are either 1 or zero depending on the size of the polygon I draw on camera.html. This is using Chrome Dev Beta Version 66.0.3346.8 on a MacBook Pro (Retina, 15-inch, Mid 2014). Camera is a Logitech BRIO via USB.

I ran some basic profiling in Chrome and as expected it looks like a lot of time is spent in the compiled opencv code.

I'm wondering if I need to write some code that uses WebGL to analyze the frame or if there's another simpler approach here that will work.

Has much effort been put into performance yet? Any low hanging fruit I can help investigate?

[janpaul123]

@osnr is currently looking into doing the whole dot detection in shaders instead of in OpenCV.

For a quick fix you could consider limiting the resolution of the camera. 4k is probably overkill for now. I'd happily accept a PR that does that (and we can revert it when performance is good enough to support that resolution).

[ajbouh]

Have you played around with minimum viable resolution for dot detection yet? I don't have a robust setup for testing various distances or lighting conditions so I'd prefer to avoid picking values that only work for me!

On Mon, Feb 19, 2018 at 5:38 PM Jan Paul Posma notifications@github.com wrote:

@osnr https://github.com/osnr is currently looking into doing the whole dot detection in shaders instead of in OpenCV.

For a quick fix you could consider limiting the resolution of the camera. 4k is probably overkill for now. I'd happily accept a PR that does that (and we can revert it when performance is good enough to support that resolution).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/janpaul123/paperprograms/issues/20#issuecomment-366843170, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAcndxHSlknr2s-evLL4NhuJ7So81_dks5tWiIcgaJpZM4SLSQ5 .

[osnr]

Another tip from JP: keep the camera window as a front tab in its window (I think keeping it focused might even be better?) Chrome will throttle background tabs and kill responsiveness.

Minimum viable resolution will definitely depend on camera and distance from papers, so it should probably be a slider or global constant or something.

[janpaul123]

I think my camera (Logitech C922) is 1280x720, which works fine.

@osnr would it be useful to share your progress so far if @ajbouh wants to play with it?

[osnr]

@ajbouh I just got this working, see this branch: https://github.com/janpaul123/paperprograms/tree/osnr-regl-detect

Neither JP nor I have a 4K webcam, so I'd love to hear how well this works for you. I get about a 3x speedup for my 1080p camera.

[janpaul123]

Posted here now: https://github.com/janpaul123/paperprograms/pull/28