Tensorflow / GPU support?

[DonFlymoor]

This is a great, inexpensive solution for full-body tracking in VR, and the only one I've found that is open source and runs with one camera. Unfortunately, VR applications already tax my CPU to the limit, and adding Media pipe to the mix makes games unplayable. A solution to that would be to use TensorFlow to inference an NVIDIA GPU, like this guide specifies:

https://google.github.io/mediapipe/getting_started/gpu_support.html

[ju1ce]

Problem with GPU support is that its quite a pain to setup in Windows. While the CPU version can be nicely packaged into an executable that works for everyone, doing that for the GPU version would be nearly impossible.

However, all the non-compiled python scripts are also availible to you on the github, so you can always try to setup your own python enviroment that supports Mediapipe GPU and run the scripts with that.

[HFede]

This is a great, inexpensive solution for full-body tracking in VR, and the only one I've found that is open source and runs with one camera. Unfortunately, VR applications already tax my CPU to the limit, and adding Media pipe to the mix makes games unplayable. A solution to that would be to use TensorFlow to inference an NVIDIA GPU, like this guide specifies:

https://google.github.io/mediapipe/getting_started/gpu_support.html

I bet a lot of people would even follow a complex setup to get better and faster tracking with only one webcam, i feel like this program has a ton of potential to be really popular if it will support TensorFlow.

[ju1ce]

Sadly, its complicated enough where you have to know what you are doing, simply following instructions isnt enough. I work with neural networks on linux quite a bit, and I still gave up trying to setup tensorflow GPU on windows cause it just refused to work for me.

For people who know how to use tensorflow and cuda, following the tutorial in the first comment should be enough to get this program woring on GPU as well.

[DonFlymoor]

I can see how it would be difficult to get running on Windows with TF (I had a lot of trouble setting it up myself), but somebody here got mediapipe working with ANGLES on Windows. https://github.com/google/mediapipe/issues/617

Would it be feasible to implement that here?

[ju1ce]

Interesting, I am not familiar with ANGLES myself. It may be possible, but I think the best way to implement GPU support would be using the JavaScript API, if I ever get to doing that. Since that would then support stuff like phones as well.

[DonFlymoor]

With the addition of Windows Subsystem for Linux, linux programs can now access your GPU on windows. This means we can run GPU accelerated mediapipe on windows without too much of a hassle, other then getting the camera working (using a phone camera wouldn't be a problem, since WSL has access to your network). If you have tensor cores, those could be levered without affecting your game's performance much.

[DonFlymoor]

Using Docker installed with the WSL 2 option, you can simply install the GPU accelerated container for mediapipe and hook your program into that. Simple, native performance, and full GPU acceleration