search

locaal-ai / obs-backgroundremoval

An OBS plugin for removing background in portrait images (video), making it easy to replace the background when recording or streaming.
GNU General Public License v2.0
2.96k stars 202 forks source link

Linux (Ubuntu 21.10) GPU Instructions #56

Closed mproffitt closed 1 year ago

mproffitt commented 3 years ago

I haven't seen anything in the documentation on this, and the code is geared around disabling GPU on any platform other than Windows but I've been playing around with this plugin today and was able to get this working with CUDA on Ubuntu Studio 21.10

Use this at your own risk - I'm not responsible if you break your card.

To enable this, you need to patch 3 files:

Patch file

gpu-linux.diff.txt

Apply the patch file with git apply gpu-linux.diff.txt

onnxruntime

Linking needs to be with the GPU version of onnx - you can find the 1.7.0 release of this here: https://github.com/microsoft/onnxruntime/releases/download/v1.7.0/onnxruntime-linux-x64-gpu-1.7.0.tgz

Alternatively the 1.9.0 release also seems to work and can be found here: https://github.com/microsoft/onnxruntime/releases/download/v1.9.0/onnxruntime-linux-x64-gpu-1.9.0.tgz

If you're using the GPU version of onnxruntime, you can't seem to switch between CPU and GPU - I haven't looked to see if this is possible, nor am I likely to try.

Building

The build procedure is the same as already documented

mkdir build && cd build
cmake .. -DWITH_CUDA=ON && cmake --build . && sudo cmake --install .

Install (Ubuntu bundled, not PPA):

mkdir -p ~/.config/obs-studio/plugins/obs-backgroundremoval/{bin,data}
cp /usr/local/lib/obs-plugins/obs-backgroundremoval.so ~/.config/obs-studio/plugins/obs-backgroundremoval/bin
cp -r /usr/local/share/obs/obs-plugins/obs-backgroundremoval/* ~/.config/obs-studio/plugins/obs-backgroundremoval/data

Tested with NVIDIA RTX 3070 TI

The Ubuntu PPA for OBS Studio and bundled OBS Studio both seem to work fine although the bundled version takes a little more setting up.

It seems to run on GPU, I've been running it for a few hours with a video background with the following observations:

https://user-images.githubusercontent.com/1852919/142483361-75871eba-bce5-4b0c-b9e8-5219ec2fb8e9.mp4

royshil commented 3 years ago

this is excellent stuff @mproffitt can you create a pull request? i'll clean it up for release so that it works alongside the other platforms

saif-ellafi commented 2 years ago

Thanks a lot mproffitt

But what a mess this is for Linux... unfortunately. Had to go through first this PR https://github.com/royshil/obs-backgroundremoval/pull/57 In order to be able to build this one. But then the plugin does not show up in OBS for Ubuntu Studio, apparently only works on OBS Studio 27.1.3? The one with Ubuntu Studio is 27.0.1 I believe... so installing OBS Studio from the PPA removes libobs0 because well the libs in Ubuntu are outdated for that. Even so, I then get a black screen when enabling the background remover...

Guess I'll have to switch to a rolling release distro. Otherwise I'll have to compile obs-studio myself. Or switch to windows...

Dang...

ghost commented 2 years ago

this is excellent stuff @mproffitt can you create a pull request? i'll clean it up for release so that it works alongside the other platforms

@royshil #57 is the PR you requested. I was able to build this and run with a GPU on Ubuntu. If you still need time to review other platforms can we update the README in the meantime to let people know GPU works on linux with this PR. If you have already tested the other platforms can we merge #57 to main?

ngoonee commented 2 years ago

Am so glad this has finally been merged. It works fine with 1.7.0, but 1.9.0 requires the includes for cuda_provider_factory.h to be removed from src/background-filter.cpp and src/Model.h

Also as previously mentioned, need to link it to the gpu version of onxxruntime. Selecting CPU or GPU doesn't do anything because it will always use GPU. https://github.com/royshil/obs-backgroundremoval/pull/57 has some ideas on how to solve this but I have not tried them out as I can't imagine WANTING to use CPU...

Mte90 commented 2 years ago

I did also that change in background-filter.cpp on line 190 as on Debian the path is different and the obs_find_module return an empty string

    std::string path = "/usr/lib/x86_64-linux-gnu/obs-plugins/obs-backgroundremoval/";
    path += tf->modelSelection.c_str();
    char* modelFilepath_rawPtr = &path[0];

I also compile everything with those commands:

cd build
cmake -DWITH_CUDA=ON .. && cmake --build . && cmake --install .

mv /usr/local/lib/obs-plugins/obs-backgroundremoval.so /usr/lib/x86_64-linux-gnu/obs-plugins/obs-backgroundremoval.so
rm -fr '/usr/lib/x86_64-linux-gnu/obs-plugins/obs-backgroundremoval'
mv /usr/local/share/obs/obs-plugins/obs-backgroundremoval/ /usr/lib/x86_64-linux-gnu/obs-plugins/
Maikuh commented 2 years ago

Seems to not be working anymore. the git apply command fails; looks like the repo has changed since then and I'm just getting a black screen when enabling the filter.

Mte90 commented 2 years ago

I had to the blank filter and it was on my tests a wrong compile because wasn't able to find the files, my patch above fix that issue forcing the right path

ngoonee commented 1 year ago

This is no longer working with 0.5.1 (which makes sense as the release does not support CUDA). Will open a separate issue for build problems I'm experiencing.

grinco commented 1 year ago

This is related: https://github.com/royshil/obs-backgroundremoval/issues/242

umireon commented 1 year ago

Now we have stable CUDA support on Linux so can we close this issue as completed? @royshil

royshil commented 1 year ago

Sure. if anything else CUDA & Linux related pops up we can handle directly