Just curious: why use the Javascript version of mediapipe instead of Python?

[liam-k]

Hi, I just found this library and first of all, it looks awesome in terms of ease of use!

But since I have been dabbling in MediaPipe a bit and I'm currently trying to decide on the best approach for my project, I'd just be interested in why you chose to go with the JS npm version of MediaPipe for this project. Wouldn't the Python version of mediapipe be somewhat more intuitive or the more obvious option, given TouchDesigner's Python integration? Curious about the thoughts behind this decision and potential downsides you are already aware of. Thank you!

[domisjustanumber]

Hey @liam-k some fine questions indeed!

There main reason is that the Python version of MediaPipe doesn't have GPU support on Windows and Mac, and even getting it working in the Linux version is a bit of a pain in the butt (i.e. you have to compile the library from scratch, the pip package is CPU only).

As we can embed the JS version of MediaPipe in an embedded Chromium instance inside TouchDesigner, we can run it all as a self-contained tox without needing anything else installed. It's definitely an annoying bit of overhead having to run Chromium and do the ML side of things in JS, but we got it working :)

I'm also doing a lot of tinkering with other Python-based ML libraries and even though they do run in Python and you'd think it would make sense to use TouchDesigner's Python abilities to run them, you quickly run into issues. So far I've found fun things like:

• The ML library need a different version of Python to the one built into TD
• The ML library needs different versions of the libraries that are part of TD's Python install
• The ML Python library entirely blocks the main TD process thread in ways that are either difficult or impossible to work around with async

In that scenario you end up needing to run a separate venv or conda env of Python just to run your ML model and still need to use NDI or Spout or other means to get the video and audio back and forth between Python and TD. dotsimulate has put together an initial pipeline to do that with his cool Stream Diffusion plugin, so worth checking out if you want to tinker there.

[liam-k]

Awesome, thank you for that detailed answer – I think you just saved me tons of potential headache (in addition to the time this whole library already saves)! Weirdly wasn’t aware of the lack of GPU support on ... basically any platform for the Python version, haha.

Seems like this lib is the way to go for me. Thanks again!