search

DBraun / PyTorchTOP

GPU PyTorch TOP in TouchDesigner with CUDA-enabled OpenCV
Other
77 stars 5 forks source link

No compositing visible in GLSL TOP #13

Closed jBachalo closed 3 years ago

jBachalo commented 3 years ago

First thanks David for your patience in getting this up and running. Spent the day compiling OpenCV and libtorch. Everything compiled without errors and copied to my Plugins folder. Inside the .toe network configured the C++ TOP to target a 32 bit model file and also set my Bytes Per Model Input Channel to 4. Toggled the 'Unload Plugin' switch.

I am feeding a live webcam video to the foreground input and using the default static cached image (candy) for the background.

However, all I see in the GLSLMultiTOP preview is the foreground live video. I assume that this is a simple compositing operation. But no alpha is being created in the foreground video. If there is a recognizable person, shouldn't compositing automatically take place?

Is there a way to check if your hardware (ie GPU) is compatible with the framework? Not running actual RTX card but RTX compatible (twin Titan Volta GPUs) which work for RTX based Ray Tracing stuff.

Any troubleshooting steps welcome and thanks again for sharing this amazing resource.

DBraun commented 3 years ago

The second input to the CPlusPlusTOP shouldn’t be the candy image. It should be a snapshot of the scene without any people in it. I crouch below my desk and use the GUI button which you can see if you go into perform mode. You will have varying quality based on the brightness/contrast of the image, whether you use homography (it should help), and sampling mode. More suggestions are here https://github.com/PeterL1n/BackgroundMattingV2/blob/master/doc/model_usage.md I’m not sure about your question but nvidia-smi a command line utility comes to mind. It has many features but I don’t use it much.

jBachalo commented 3 years ago

Making more sense. 2nd input is the calibration or reference image. Loaded a different model, re-inited. Finally seeing a matte image(against alpha) in C++TOP preview!! but the actual matte is static. Must be getting close. Will play some more. Finally. Working with torchscript_resnet50_fp32 model. Also Un load Plugin I had switched to 'On' Awesome!