torinmb / mediapipe-touchdesigner

GPU Accelerated MediaPipe Plugin for TouchDesigner
MIT License
637 stars 31 forks source link

Webcam initalization assumption #60

Closed joreg closed 7 months ago

joreg commented 9 months ago

hey dom and torin,

i see an issue where on some machines the webpage would run and show the camera but the prediction-loop is not initialzed. i tracked this down to what i believe is a wrong assumption in your setup() routine. i am not really familiar with js so maybe i'm reading things wrong, but here goes:

i wonder if instead, at the end of setup() we don't start the camera but always start the prediction-loop. in the loop we check if the camera is running. so whenever the camera is then started or changed, prediction would start?!

domisjustanumber commented 9 months ago

Hey @joreg thanks for taking the time to dig into this. Getting webcams to reliably start and return data has been something I've been tinkering with for a while, and this is a new way of it breaking that I haven't seen before!

That's a good idea on how to catch it - I can move the webcam started detection into the prediction loop - then if the webcam takes a while to start, there are more chances to catch it later.

If you want to make the change, feel free to submit a PR for it and/or I'll be doing some work on this project over the holidays so can put the change in then.

joreg commented 9 months ago

so here is how i fixed it on our end for now: https://github.com/vvvv/VL.MediaPipe/commit/0b56dff667e591bdd1162a92fd1414506830d465?diff=unified&w=1

not sure if this is the best way, but it seems to work so far.

domisjustanumber commented 9 months ago

Oh amazing, thank you!

deseipel commented 9 months ago

I'm also having web cam issues when running it. Seems like the camera its trying to use doesn't default to the correct resolution. But I'm not sure how I can set that in the MediaPipe container. And the image segmentation container is supposed to take the same cam input? I did come up with a workaround using NDI in/out.

domisjustanumber commented 7 months ago

Hey @joreg I rolled some of the suggested changes you made into the latest release, so hopefully the webcam loads more reliably and faster now. Thanks for the tip and you're welcome to copy any changes to your vvvv implementation!

https://github.com/torinmb/mediapipe-touchdesigner/releases/tag/v0.4.1

domisjustanumber commented 7 months ago

@deseipel this release should also (hopefully) fix your webcam startup/resolution issue https://github.com/torinmb/mediapipe-touchdesigner/releases/tag/v0.4.1

joreg commented 7 months ago

@domisjustanumber thanks! i've just updated our pack with your latest code. no more custom modifications on our end.