search

rayburgemeestre / webcam-virtual-background

Linux Virtual Background Webcam
Mozilla Public License 2.0
36 stars 2 forks source link

I'm getting this error when I try to compile from source. My distro is Debian Sid #3

Open V1ru5-99 opened 1 year ago

V1ru5-99 commented 1 year ago

make compile

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/ffmpeg/lib:$PWD/build/tensorflow/bazel-bin/tensorflow/lite \

PKG_CONFIG_PATH=$PWD/ffmpeg/lib/pkgconfig c++ -O0 -g --std=c++17 -I$PWD/ffmpeg/include -I$PWD/build/tensorflow/ -I$PWD/build/tensorflow/third_party/ \

PKG_CONFIG_PATH=$PWD/ffmpeg/lib/pkgconfig c++ -O2 --std=c++17 -I$PWD/ffmpeg/include -I$PWD/build/tensorflow/ -I$PWD/build/tensorflow/third_party/ \ -I$PWD/ffmpeg-4.4 \ -I$PWD/build/mediapipe \ -I$PWD/build/tensorflow/tensorflow/lite/tools/make/downloads/flatbuffers/include \ -I$PWD/build/cpp-readline/src \ -I$PWD/build/tiny-process-library \ -L$PWD/ffmpeg/lib \ -L$PWD/build/tensorflow/bazel-bin/tensorflow/lite \ -Wl,-rpath=lib \ src/main.cpp src/transpose_conv_bias.cc src/blur_float.cpp build/cpp-readline/src/Console.cpp src/snowflake.cpp \ -lavdevice -lavformat -lavcodec -lavutil -ltensorflowlite -lswscale -lreadline -lpthread \ $PWD/build/tiny-process-library/build/libtiny-process-library.a \ -o main src/main.cpp:7:10: fatal error: Console.hpp: No such file or directory 7 | #include "Console.hpp" | ^~~~~ compilation terminated. src/transpose_conv_bias.cc:19:10: fatal error: mediapipe/util/tflite/operations/transpose_conv_bias.h: No such file or directory 19 | #include "mediapipe/util/tflite/operations/transpose_conv_bias.h" | ^~~~~~~~~~~~ compilation terminated. cc1plus: fatal error: build/cpp-readline/src/Console.cpp: No such file or directory compilation terminated. make: *** [Makefile:66: compile] Error 1

V1ru5-99 commented 1 year ago

@rayburgemeestre Can you help me please?

rayburgemeestre commented 1 year ago

Well, I recall that it was quite a bit of hell for me to get it to compile a while ago. :joy: Did you run make configure first though? It should "configure" a bunch of dependencies (it actually also compiles a lot of stuff) Then the make compile target (if the previous step succeeded) should be able to find those missing header files that you are seeing

rayburgemeestre commented 1 year ago

I haven't tested this myself, at least not recently, but if you have docker on your system, it might be an option for you to try: make build-shell, it should start up a build container, that someone else used to develop a feature (it was contributed) I did build the container and pushed it to dockerhub, so you don't have to build it yourself, but I don't remember if I used it to compile the project from scratch successfully or not

V1ru5-99 commented 1 year ago

@rayburgemeestre ok, so this is kinda weird, I did run "make configure" before but nothing changed, still same error, when I tried "make build-shell" i get this: docker run -it -v $PWD:$PWD -w $PWD rayburgemeestre/virtual-bg-dev-container:latest /bin/bash /bin/bash: line 1: docker: command not found make: *** [Makefile:142: build-shell] Error 127 However when I tried to install docker it tells me that I already have it installed (??):

sudo apt install docker

Reading package lists... Done Building dependency tree... Done Reading state information... Done docker is already the newest version (1.5-2). The following packages were automatically installed and are no longer required: g++-11 gir1.2-javascriptcoregtk-4.0 gir1.2-soup-2.4 gir1.2-webkit2-4.0 ibverbs-providers libaccinj64-11.8 libatk-wrapper-java libatk-wrapper-java-jni libbabeltrace1 libboost-regex1.74.0 libcu++-dev libcub-dev libcublas11 libcublaslt11 libcudart11.0 libcufft10 libcufftw10 libcupti-dev libcupti-doc libcupti11.8 libcurand10 libcusolver11 libcusolvermg11 libcusparse11 libdebuginfod-common libdebuginfod1 libdmapsharing-3.0-2 libgl-dev libglx-dev libhwloc-plugins libhwloc15 libibumad3 libibverbs1 libipt2 libjs-jquery libnppc11 libnppial11 libnppicc11 libnppidei11 libnppif11 libnppig11 libnppim11 libnppist11 libnppisu11 libnppitc11 libnpps11 libnvblas11 libnvidia-ptxjitcompiler1 libnvjpeg11 libnvrtc-builtins11.8 libnvrtc11.2 libnvtoolsext1 libnvvm4 libpfm4 libpthread-stubs0-dev librdmacm1 libsoup-gnome2.4-1 libsource-highlight-common libsource-highlight4v5 libssh-4 libstdc++-11-dev libtbb-dev libtbb12 libtbbbind-2-5 libtbbmalloc2 libthrust-dev libucx0 libvdpau-dev libx11-dev libxau-dev libxcb1-dev libxdmcp-dev libxnvctrl0 nsight-compute nsight-compute-target nvidia-cuda-gdb nvidia-cuda-toolkit-doc nvidia-legacy-check nvidia-opencl-dev nvidia-openjdk-8-jre ocl-icd-opencl-dev opencl-c-headers opencl-clhpp-headers x11proto-dev xorg-sgml-doctools xtrans-dev Use 'sudo apt autoremove' to remove them. 0 upgraded, 0 newly installed, 0 to remove and 252 not upgraded.

V1ru5-99 commented 1 year ago

@rayburgemeestre Meanwhile can you just upload a compiled version of the program on releases?

ChillAndImprove commented 1 year ago

@rayburgemeestre ok, so this is kinda weird, I did run "make configure" before but nothing changed, still same error, when I tried "make build-shell" i get this: docker run -it -v $PWD:$PWD -w $PWD rayburgemeestre/virtual-bg-dev-container:latest /bin/bash /bin/bash: line 1: docker: command not found make: *** [Makefile:142: build-shell] Error 127 However when I tried to install docker it tells me that I already have it installed (??):

sudo apt install docker

Reading package lists... Done Building dependency tree... Done Reading state information... Done docker is already the newest version (1.5-2). The following packages were automatically installed and are no longer required: g++-11 gir1.2-javascriptcoregtk-4.0 gir1.2-soup-2.4 gir1.2-webkit2-4.0 ibverbs-providers libaccinj64-11.8 libatk-wrapper-java libatk-wrapper-java-jni libbabeltrace1 libboost-regex1.74.0 libcu++-dev libcub-dev libcublas11 libcublaslt11 libcudart11.0 libcufft10 libcufftw10 libcupti-dev libcupti-doc libcupti11.8 libcurand10 libcusolver11 libcusolvermg11 libcusparse11 libdebuginfod-common libdebuginfod1 libdmapsharing-3.0-2 libgl-dev libglx-dev libhwloc-plugins libhwloc15 libibumad3 libibverbs1 libipt2 libjs-jquery libnppc11 libnppial11 libnppicc11 libnppidei11 libnppif11 libnppig11 libnppim11 libnppist11 libnppisu11 libnppitc11 libnpps11 libnvblas11 libnvidia-ptxjitcompiler1 libnvjpeg11 libnvrtc-builtins11.8 libnvrtc11.2 libnvtoolsext1 libnvvm4 libpfm4 libpthread-stubs0-dev librdmacm1 libsoup-gnome2.4-1 libsource-highlight-common libsource-highlight4v5 libssh-4 libstdc++-11-dev libtbb-dev libtbb12 libtbbbind-2-5 libtbbmalloc2 libthrust-dev libucx0 libvdpau-dev libx11-dev libxau-dev libxcb1-dev libxdmcp-dev libxnvctrl0 nsight-compute nsight-compute-target nvidia-cuda-gdb nvidia-cuda-toolkit-doc nvidia-legacy-check nvidia-opencl-dev nvidia-openjdk-8-jre ocl-icd-opencl-dev opencl-c-headers opencl-clhpp-headers x11proto-dev xorg-sgml-doctools xtrans-dev Use 'sudo apt autoremove' to remove them. 0 upgraded, 0 newly installed, 0 to remove and 252 not upgraded.

Could you confirm whether you have Docker installed on your system? You can verify this by using the Docker CLI command docker --version in your terminal. If Docker is not installed, you can follow the instructions from the official Docker documentation at this link: https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository.

ChillAndImprove commented 1 year ago

@rayburgemeestre Meanwhile can you just upload a compiled version of the program on releases?

That would indeed be a great idea, but compiling the entire program statically could potentially be quite a big task.

rayburgemeestre commented 1 year ago

Hey @V1ru5-99 I wanted to reply to your message from 4 hours ago with a link, and then I discovered that actually I forgot to upload the binary. I just uploaded it here now: https://github.com/rayburgemeestre/webcam-virtual-background/releases/tag/1.1

wget https://github.com/rayburgemeestre/webcam-virtual-background/releases/download/1.1/virtual-bg-1.1.tar.gz

It was supposed to be there, sorry. I did make the tarball when I made that 1.1 release, I just forgot to upload it I guess. I tested just now if it works in debian:sid using docker (since I use Ubuntu 20.04), and I seem to get the binary working. This is how I started docker on my system:

xhost +
docker run --privileged -it -e DISPLAY=\$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $PWD:$PWD -w $PWD debian:sid /bin/bash

Just sharing the above as a 'fyi', but in your case your system is debian:sid, so no docker needed. Inside docker, or in your case on your system, I would ensure that these dependencies are installed:

apt update
apt-get install -y v4l2loopback-dkms make sudo libreadline8 libavfilter8 libxcb-shape0 ffmpeg

Then if you download the binaries from the URL I send you (extracting the virtual-bg-1.1.tar.gz) there is a 'cam' script there which invokes 'main' after creating the two virtual devices. But you will probably get this error:

root@9d3f162f2896:/home/rayb/tmp/virtual-bg-1.1# ./cam
./main: error while loading shared libraries: libavfilter.so.7: cannot open shared object file: No such file or directory

On debian:sid the version is libavfilter.so.8, but luckily I bundled that dependency in the 'lib' dir, it just seems that I forgot to put the following line in the cam script: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/lib You can edit the file and put it before the ./main line, and that should fix it. Or invoke it manually, like below:

root@9d3f162f2896:/home/rayb/tmp/virtual-bg-1.1# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/lib 
root@9d3f162f2896:/home/rayb/tmp/virtual-bg-1.1# ./cam

art by: Marcin Glinski          _
                               / \
                              / .'_
                             / __| \
             `.             | / (-' |
           `.  \_..._       :  (_,-/
         `-. `,'     `-.   /`-.__,'
            `/ __       \ /     /
            /`/  \       :'    /
          _,\o\_o/       /    /
         (_) ___.--.    /    /
          `-. -._.i \.      :
             `.\  ( |:.     |
            ,' )`-' |:..   / \
   __     ,'   |    `.:.      `.
  (_ `---:     )      \:.       \
   ,'     `. .'\       \:.       )
 ,' ,'     ,'  \\ o    |:.      /
(_,'  ,7  /     \`.__.':..     /,,,
  (_,'(_,'   _gdMbp,,dp,,,,,,dMMMMMbp,,
          ,dMMMMMMMMMMMMMMMMMMMMMMMMMMMb,
       .dMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMb,  fsc
     .dMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM,
    ,MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
   dMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM.
 .dMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMb
 V I R T U A L    B A C K G R O U N D   W E B C A M

Available commands are:
    preview
    stop
    set-mode
    set-cam
    list-cams
    start
    exit
    quit
    set-model
    run
    help
cam>

@ChillAndImprove I didn't do a fully static binary indeed, but at least I also shipped most .so libraries it needs in the lib directory in the .tar.gz file. :sweat_smile:

rayburgemeestre commented 1 year ago

I'll re-upload the .tar.gz file tomorrow with the export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD/lib fix in the 'cam' script to prevent others from running into it... :crossed_fingers: that it works on your system

V1ru5-99 commented 1 year ago

@rayburgemeestre Hi, thank you for uploading the binary, the program works just fine! How do i change the virtual camera name? I need to change it so that my browser won't detect that I'm using a virtual camera. Edit: nevermind, I figured it out myself, unfortunately the camera is a bit laggy, i get like a 1 second delay, is there anything I can do about it?

V1ru5-99 commented 1 year ago

@rayburgemeestre @ChillAndImprove How do I change the codec to improve performance? I'm currently running on nvidia 340 because I'm running with an older graphics card (GT 210)

rayburgemeestre commented 1 year ago

Hey @V1ru5-99 I haven't experienced that much lag personally, I can actually not notice much of the lag in my case.

I think it makes sense to figure out which step is introducing the lag. Let's say your camera is /dev/video0, you can do something like:

ffplay /dev/video0

It should be without 1 second lag.. since it's the source of the video, if there is a problem there already there isn't much we can do about it.

Then after you "start" the cam tool, you can try:

ffplay /dev/video8

If it is 1 second behind, the lag is introduced in initial resize + converting stage. Could you also share what the "start" command prints for the input? In my case this is:

Input #0, video4linux2,v4l2, from '':
  Duration: N/A, start: 65904.341082, bitrate: 110592 kb/s
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480, 110592 kb/s, 30 fps, 30 tbr, 1000k tbn, 1000k tbc

If there are no issues with /dev/video8, then we need to check /dev/video9...

With: ffplay /dev/video9

If that has 1 second lag, then the slowness is in the last part where the background blurring etc happens.. That might be a little bit harder to solve, so I hope it's something with the resize step.

If the slowness is in /dev/video8, then I would suggest the following debugging.. We do: /dev/video0 -> /dev/video8 to resize and convert to YUV. The command for this something like:

    /usr/bin/ffmpeg -pix_fmt mjpeg -i /dev/video0
        -f v4l2 -input_format mjpeg -framerate 10 -video_size 1024x680 -vf 
          scale=640:480:force_original_aspect_ratio=increase,crop=640:480 -pix_fmt yuv420p -f v4l2 /dev/video8;

You could run that command manually, and then view ffplay /dev/video8, and perhaps customize it until it becomes fast. Now that I looked this command up, I'm a little surprised about the hardcoded -video_size 1024x680 part in the command, and I'm tempted to see if I can get rid of it.

In my case I could do that and use:

    /usr/bin/ffmpeg -pix_fmt mjpeg -i /dev/video0 \
        -f v4l2 -input_format mjpeg -framerate 10 -vf  \
          scale=640:480:force_original_aspect_ratio=increase,crop=640:480 -pix_fmt yuv420p -f v4l2 /dev/video8;

I believe it would be interesting to see what happens if you scale it down even further, e.g.:

    /usr/bin/ffmpeg -pix_fmt mjpeg -i /dev/video0 \
        -f v4l2 -input_format mjpeg -framerate 10 -vf  \
          scale=480:320:force_original_aspect_ratio=increase,crop=480:320 -pix_fmt yuv420p -f v4l2 /dev/video8;

And then, all the way up to:

    /usr/bin/ffmpeg -pix_fmt mjpeg -i /dev/video0 \
        -f v4l2 -input_format mjpeg -framerate 10 -vf  \
          scale=320:240:force_original_aspect_ratio=increase,crop=320:240 -pix_fmt yuv420p -f v4l2 /dev/video8;

If the lag is reducing then it might be a CPU bound issue.. In the code I've hardcoded the 640x480 resolution, so it won't fix your issue but if it solves it, then I could prioritize making the resolution customizable

V1ru5-99 commented 1 year ago

@rayburgemeestre Hi, sorry for the late response, I've thought that the lag happens in /dev/video8 because my camera is running just fine without the virtual background program however now when i tried to run ffplay /dev/video0 I've noticed that the 1 second delay is there aswell. I've also tried https://volcomix.github.io/virtual-background/ and it also worked fine using Canvas 2D + CPU. So I belive there is a chance that the lag might be caused by a specific setting used by ffmpeg. Unfortunately I'm not so familiar with configuring ffmpeg in order to see what's the issue so, I'm gonna ask for your help here.

rayburgemeestre commented 1 year ago

Alright, So ffplay /dev/video0 is already laggy..

Do the following flags help? I would try something in the following order probably...

ffplay -fflags nobuffer /dev/video0
ffplay -fflags nobuffer -flags low_delay /dev/video0
ffplay -fflags nobuffer -flags low_delay -framedrop /dev/video0
ffplay -fflags nobuffer -flags low_delay -framedrop -strict experimental /dev/video0

Source: https://stackoverflow.com/a/49273163

If the flags help maybe I can simply add them by default

V1ru5-99 commented 1 year ago

@rayburgemeestre Thanks, the "nobuffer" flag alone seems to fix the issue. I haven't noticed any difference using the other flags. Let me know how do I apply the flag using your program.

V1ru5-99 commented 1 year ago

Alright, So ffplay /dev/video0 is already laggy..

Do the following flags help? I would try something in the following order probably...

ffplay -fflags nobuffer /dev/video0
ffplay -fflags nobuffer -flags low_delay /dev/video0
ffplay -fflags nobuffer -flags low_delay -framedrop /dev/video0
ffplay -fflags nobuffer -flags low_delay -framedrop -strict experimental /dev/video0

Source: https://stackoverflow.com/a/49273163

If the flags help maybe I can simply add them by default

@rayburgemeestre Hi, its been a while, can you add the nobuffer flag by default? Or how do I manually run the flag with the virtual background?

rayburgemeestre commented 1 year ago

Ok Ive made a release with this fix, hope it works :crossed_fingers: https://github.com/rayburgemeestre/webcam-virtual-background/releases/tag/1.1.1 :grin:

V1ru5-99 commented 1 year ago

Hi @rayburgemeestre, thank you for uploading the release! The lag is now gone in /dev/video8. In /dev/video9 there is a small delay of 0.1 - 0.2 seconds maybe, I was wondering if you can look into that when you have time, otherwise the background works fine, the latency is noticeable but it can also be ignored. However I'd rather see it fixed if possible :).