Closed RyanHakurei closed 2 years ago
k4yt3x
commented
2 years ago You launched way too many processes. You probably blew up your VRAM or something doing that many processes. Start with one.
You do need to lower your process count, but I guess that's not the main problem since I missed the vkEnumeratePhysicalDevices earlier. I think it's more likely that you missed the nvidia-docker2 package? What system and GPU are you using?
RyanHakurei
commented
2 years ago I do have Nvidia-docker installed, and I've confirmed it works using Nvidia's Cuda container. I am using Arch with a GTX 1070.
k4yt3x
commented
2 years ago Does this work:
docker run --rm nvidia/vulkan:1.2.133-450 vulkaninfoFunnily enough, no, which is odd because I know Vulkan works on my install as I play Doom in Vulkan mode and it runs without issue, and of course Proton/DXVK work just fine.
Running vulkaninfo bare metal works.
k4yt3x
commented
2 years ago You might be missing some packages? I'm not sure why Vulkan's not working in your container.
RyanHakurei
commented
2 years ago The thing is though, Vulkan works just fine bare metal so I am admittedly confused why it doesn't work in containers.
plambe
commented
2 years ago I think I've got the same problem. video2x fails in the same way already discussed here:
$ docker run -it --rm --gpus 1 -v $(pwd):/host ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i ttd_19_0005.mp4 -o ttd_19_0005_upscaled.mp4 -p3 upscale -h 720 -a waifu2x -n311:52:28.663216 | INFO | Video2X 5.0.0-beta5
11:52:28.663460 | INFO | Copyright (C) 2018-2022 K4YT3X and contributors.
11:52:28.663608 | INFO | Reading input video information
11:52:28.749566 | INFO | Starting video decoder
11:52:28.755136 | INFO | Starting video encoder
11:52:28.766545 | INFO | Upscaler process 0 initiating
11:52:28.769329 | INFO | Upscaler process 1 initiating
11:52:28.772603 | INFO | Upscaler process 2 initiating
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'ttd_19_0005.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.45.100
Duration: 00:00:04.00, start: 0.000000, bitrate: 1002 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x448, 998 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Output #0, rawvideo, to 'pipe:1':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Stream #0:0(und): Video: rawvideo (RGB[24] / 0x18424752), rgb24, 640x448, q=2-31, 172032 kb/s, 25 fps, 25 tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc58.54.100 rawvideo
11:52:29.772962 | ERROR | process died unexpectedly
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
| | -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free
software: you...
| -> <code object <module> at 0x7f7d2c868f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
-> <function _run_code at 0x7f7d3c2c8040>
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
| -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free software:
you...
-> <code object <module> at 0x7f7d2c868f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
File "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 30, in <module>
sys.exit(main())
| | -> <function main at 0x7f7d2c86e040>
| -> <built-in function exit>
-> <module 'sys' (built-in)>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 612, in main
video2x.upscale(
| -> <function Video2X.upscale at 0x7f7d2c903ee0>
-> <video2x.video2x.Video2X object at 0x7f7d2c875160>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 412, in upscale
self._run(
| -> <function Video2X._run at 0x7f7d2c903e50>
-> <video2x.video2x.Video2X object at 0x7f7d2c875160>
> File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 308, in _run
raise Exception("process died unexpectedly")
Exception: process died unexpectedly
11:52:29.786161 | INFO | Stopping processor processes
11:52:29.786858 | INFO | Stopping decoder and encoder threads
11:52:29.882654 | DEBUG | Decoding queue depleted
11:52:29.886251 | DEBUG | Encoding queue depleted
11:52:30.261340 | INFO | Decoder thread exiting
11:52:30.262984 | INFO | Encoder thread exiting
11:52:30.263661 | ERROR | process died unexpectedly
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
| | -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free
software: you...
| -> <code object <module> at 0x7f7d2c868f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
-> <function _run_code at 0x7f7d3c2c8040>
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
| -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free software:
you...
-> <code object <module> at 0x7f7d2c868f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
File "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 30, in <module>
sys.exit(main())
| | -> <function main at 0x7f7d2c86e040>
| -> <built-in function exit>
-> <module 'sys' (built-in)>
> File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 612, in main
video2x.upscale(
| -> <function Video2X.upscale at 0x7f7d2c903ee0>
-> <video2x.video2x.Video2X object at 0x7f7d2c875160>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 412, in upscale
self._run(
| -> <function Video2X._run at 0x7f7d2c903e50>
-> <video2x.video2x.Video2X object at 0x7f7d2c875160>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 375, in _run
raise exception[0]
-> [Exception('process died unexpectedly')]
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 308, in _run
raise Exception("process died unexpectedly")
Exception: process died unexpectedlyvulkaninfo also fails in a docker container:
$ docker run --rm --gpus 1 nvidia/vulkan:1.2.133-450 vulkaninfo ERROR: [Loader Message] Code 0 : loader_scanned_icd_add: Could not get 'vkCreateInstance' via 'vk_icdGetInstanceProcAddr' for ICD libGLX_nvidia.so.0
Cannot create Vulkan instance.
This problem is often caused by a faulty installation of the Vulkan driver or attempting to use a GPU that does not support Vulkan.
/root/sdk-build/1.2.131.2/source/Vulkan-Tools/vulkaninfo/vulkaninfo.h:371: failed with ERROR_INCOMPATIBLE_DRIVERIf I add a few devices, vulkaninfo would start working:
$ docker run --rm --gpus 1 --device /dev/nvidia0 --device /dev/nvidia-uvm --device /dev/nvidia-uvm-tools --device /dev/nvidiactl nvidia/vulkan:1.2.133-450 vulkaninfo | head -5
'DISPLAY' environment variable not set... skipping surface info
error: XDG_RUNTIME_DIR not set in the environment.
==========
VULKANINFO
==========
Vulkan Instance Version: 1.2.131video2x with the same mounted devices still fails though:
$ docker run -it --rm --gpus 1 --device /dev/nvidia0 --device /dev/nvidia-uvm --device /dev/nvidia-uvm-tools --device /dev/nvidiactl -v $(pwd):/host ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i ttd_19_0005.mp4 -o ttd_19_0005_upscaled.mp4 -p3 upscale -h 720 -a waifu2x -n3
11:58:35.388275 | INFO | Video2X 5.0.0-beta5
11:58:35.388532 | INFO | Copyright (C) 2018-2022 K4YT3X and contributors.
11:58:35.388680 | INFO | Reading input video information
11:58:35.479285 | INFO | Starting video decoder
11:58:35.486704 | INFO | Starting video encoder
11:58:35.497250 | INFO | Upscaler process 0 initiating
11:58:35.502928 | INFO | Upscaler process 1 initiating
11:58:35.502984 | INFO | Upscaler process 2 initiating
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'ttd_19_0005.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.45.100
Duration: 00:00:04.00, start: 0.000000, bitrate: 1002 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x448, 998 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Output #0, rawvideo, to 'pipe:1':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Stream #0:0(und): Video: rawvideo (RGB[24] / 0x18424752), rgb24, 640x448, q=2-31, 172032 kb/s, 25 fps, 25 tbn, 25 tbc (default)
Metadata:
handler_name : VideoHandler
encoder : Lavc58.54.100 rawvideo
11:58:36.502536 | ERROR | process died unexpectedly
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
| | -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free
software: you...
| -> <code object <module> at 0x7fac3e8f5f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
-> <function _run_code at 0x7fac4e355040>
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
| -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free software:
you...
-> <code object <module> at 0x7fac3e8f5f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
File "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 30, in <module>
sys.exit(main())
| | -> <function main at 0x7fac3e8fb040>
| -> <built-in function exit>
-> <module 'sys' (built-in)>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 612, in main
video2x.upscale(
| -> <function Video2X.upscale at 0x7fac3e990ee0>
-> <video2x.video2x.Video2X object at 0x7fac3e9021f0>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 412, in upscale
self._run(
| -> <function Video2X._run at 0x7fac3e990e50>
-> <video2x.video2x.Video2X object at 0x7fac3e9021f0>
> File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 308, in _run
raise Exception("process died unexpectedly")
Exception: process died unexpectedly
11:58:36.515682 | INFO | Stopping processor processes
11:58:36.516599 | INFO | Stopping decoder and encoder threads
11:58:36.615740 | DEBUG | Decoding queue depleted
11:58:36.616681 | DEBUG | Encoding queue depleted
11:58:36.990855 | INFO | Decoder thread exiting
11:58:36.993572 | INFO | Encoder thread exiting
11:58:36.994416 | ERROR | process died unexpectedly
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
| | -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free
software: you...
| -> <code object <module> at 0x7fac3e8f5f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
-> <function _run_code at 0x7fac4e355040>
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
| -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free software:
you...
-> <code object <module> at 0x7fac3e8f5f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
File "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 30, in <module>
sys.exit(main())
| | -> <function main at 0x7fac3e8fb040>
| -> <built-in function exit>
-> <module 'sys' (built-in)>
> File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 612, in main
video2x.upscale(
| -> <function Video2X.upscale at 0x7fac3e990ee0>
-> <video2x.video2x.Video2X object at 0x7fac3e9021f0>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 412, in upscale
self._run(
| -> <function Video2X._run at 0x7fac3e990e50>
-> <video2x.video2x.Video2X object at 0x7fac3e9021f0>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 375, in _run
raise exception[0]
-> [Exception('process died unexpectedly')]
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 308, in _run
raise Exception("process died unexpectedly")
Exception: process died unexpectedlyI suppose I have to mount some more stuff and it will probably start working, but at the moment I don't know what.
I'm on a devuan chimaera (based on debian bullseye) with Nvidia driver version 460.91.03 and nvidia-docker works fine:
$ docker run --rm --gpus all --device /dev/nvidia0 --device /dev/nvidia-uvm --device /dev/nvidia-uvm-tools --device /dev/nvidiactl nvidia/cuda:11.0-base nvidia-smi | head -3 | tail -1
| NVIDIA-SMI 460.91.03 Driver Version: 460.91.03 CUDA Version: 11.2 |NVIDIA GPUs work for me without any mounts or --device. I only had to add --device /dev/dri for AMD GPUs. I'm honestly not sure what's wrong either since ncnn only says:
vkEnumeratePhysicalDevices failed -3
...not a lot of useful information to work with. Usually when this happens it's caused by a missing driver like libvulkan0 or cuda-drivers, but both should be already installed in the container so I'm not sure what else might be missing.
AChmykhov
commented
2 years ago Vulkan needs X server and display and stuff to properly run. I've found this on nvidia discussion forum, but there is probable something in documentation: https://forums.developer.nvidia.com/t/getting-vk-error-incompatible-driver/45948/2
I've added line to Dockerfile to install xserver-xorg-core xserver-xorg xorg openbox xserver-xorg-video-dummy and tmux (i'll explain later), I've also added to the image a dummy screen configuration file.
I cannot figure out way to run X -config /dummy.conf in detached mode, so i don't run this command in Dockerfile (so while building image) or just in container's bash. I run this in separate pane in tmux, and then, in the other pane i run export DISPLAY=:0 and /usr/bin/python3.8 -m video2x -i input.mp4 -o output.mp4 -p4 upscale -h 2160 -a srmd -n4. But this also don't work, with similar error.
I'm trying to go along this tutorial as much as i can: https://techoverflow.net/2019/02/23/how-to-run-x-server-using-xserver-xorg-video-dummy-driver-on-ubuntu/ but i'm a little new to working with X server, so i do not know, where are my mistakes.
Hope it helps :)
k4yt3x
commented
2 years ago I know older nvidia drivers requires X server to run, but that should no longer be required right now. Vulkan shouldn't need X server either. I've also been running it without X server on my workstation and it works just fine.
k4yt3x
commented
2 years ago I did a test on one of our community member (Mason)'s machine, and all I had to install were nvidia, nvidia-docker, and podman for it to work. I'm still unable to reproduce this issue. I'm leaning towards it being a problem specific to your system?
P.S. ctt is just an alias of sudo podman run -it --rm -h=temp --name=temp
k4yt3x
commented
2 years ago Oh wait I got the exact same problem with docker, but it works fine with podman. Try podman?
plambe
commented
2 years ago I had to install podman to test this. It didn't work, but with a podman-related error.
$ podman run -it --rm -v $(pwd):/host ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i TimeToDisco\(45\).mp4 -o TimeToDisco\(45\)_upscaled.mp4 -p3 upscale -h 720 -a waifu2x -n3
ERRO[0000] unable to write pod event: "write unixgram @0004c->/run/systemd/journal/socket: sendmsg: no such file or directory"
ERRO[0000] unable to write pod event: "write unixgram @0004c->/run/systemd/journal/socket: sendmsg: no such file or directory"
Error: container_linux.go:367: starting container process caused: error adding seccomp filter rule for syscall bdflush: permission denied: OCI permission deniedI'll see what I can do about that later.
plambe
commented
2 years ago Got it to run unexpectedly fast, now I get the same error as with docker
$ podman run -it --rm -v $(pwd):/host ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i TimeToDisco\(45\).mp4 -o TimeToDisco\(45\)_upscaled.mp4 -p3 upscale -h 720 -a waifu2x -n3
ERRO[0000] unable to write pod event: "write unixgram @0004f->/run/systemd/journal/socket: sendmsg: no such file or directory"
ERRO[0000] unable to write pod event: "write unixgram @0004f->/run/systemd/journal/socket: sendmsg: no such file or directory"
ERRO[0000] unable to write pod event: "write unixgram @0004f->/run/systemd/journal/socket: sendmsg: no such file or directory"
ERRO[0000] unable to write pod event: "write unixgram @0004f->/run/systemd/journal/socket: sendmsg: no such file or directory"
09:03:26.175714 | INFO | Video2X 5.0.0-beta5
09:03:26.175987 | INFO | Copyright (C) 2018-2022 K4YT3X and contributors.
09:03:26.176130 | INFO | Reading input video information
09:03:26.382027 | INFO | Starting video decoder
09:03:26.389053 | INFO | Starting video encoder
09:03:26.397588 | INFO | Upscaler process 0 initiating
09:03:26.399667 | INFO | Upscaler process 1 initiating
09:03:26.402767 | INFO | Upscaler process 2 initiating
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
[..]It's very possible the issue is with my system, as it's devuan, which lacks systemd and this makes it quite different. Nevertheless, I'd be quite happy if I get this working.
RyanHakurei
commented
2 years ago I did a test on one of our community member (Mason)'s machine, and all I had to install were
nvidia,nvidia-docker, andpodmanfor it to work. I'm still unable to reproduce this issue. I'm leaning towards it being a problem specific to your system?
P.S.
cttis just an alias ofsudo podman run -it --rm -h=temp --name=temp
Let me try with Podman, I switched to Docker as I precisely didn't expect Nvidia-Docker to work with Podman but apparently Podman has come a long way since I last used it.
Update: Running with Podman seems to work. @hazrpg try with Podman, on Arch at least Podman seems to work whereas Docker does not.
Update 2: Actually Docker seems to work too now. ¯\_(ツ)_/¯
hazrpg
commented
2 years ago I'm getting the same issue in Ubuntu 20.04 LTS, all the previous attempts gets me closer... but not by much, the code still errors out. Not sure what else to try really.
k4yt3x
commented
2 years ago Update 2: Actually Docker seems to work too now. ¯_(ツ)_/¯
k4yt3x
commented
2 years ago @plambe I think your errors might be caused by insufficient permissions.
plambe
commented
2 years ago Well, I'm willing to experiment. Any suggestions on what permissions should I grant, etc?
k4yt3x
commented
2 years ago @plambe I have concerns with kernel.unprivileged_userns_clone, which is used by Podman to allow non-root users to run Podman containers, since it increases attack surface on the system. It had to be enabled in a recent k8s LPE vulnerability.... Anyways, what I'm trying to say is: try sudo podman.
plambe
commented
2 years ago So I ran it as root, which required some conf file (/etc/containers/containers.conf) to be created and it fails in the same way:
# podman run -it --rm -v $(pwd):/host ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i TimeToDisco\(45\).mp4 -o TimeToDisco\(45\)_upscaled.mp4 -p3 upscale -h 720 -a waifu2x -n3
ERRO[0000] unable to write pod event: "write unixgram @00075->/run/systemd/journal/socket: sendmsg: no such file or directory"
WARN[0000] Failed to add conmon to systemd sandbox cgroup: The name org.freedesktop.systemd1 was not provided by any .service files
ERRO[0000] unable to write pod event: "write unixgram @00075->/run/systemd/journal/socket: sendmsg: no such file or directory"
Error: OCI runtime error: sd-bus call: No route to host
# vi /etc/containers/containers.conf
# cat /etc/containers/containers.conf
[engine]
cgroup_manager = "cgroupfs"
# podman run -it --rm -v $(pwd):/host ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i TimeToDisco\(45\).mp4 -o TimeToDisco\(45\)_upscaled.mp4 -p3 upscale -h 720 -a waifu2x -n3
ERRO[0000] unable to write pod event: "write unixgram @00079->/run/systemd/journal/socket: sendmsg: no such file or directory"
ERRO[0000] unable to write pod event: "write unixgram @00079->/run/systemd/journal/socket: sendmsg: no such file or directory"
ERRO[0000] unable to write pod event: "write unixgram @00079->/run/systemd/journal/socket: sendmsg: no such file or directory"
ERRO[0000] unable to write pod event: "write unixgram @00079->/run/systemd/journal/socket: sendmsg: no such file or directory"
17:19:23.384765 | INFO | Video2X 5.0.0-beta5
17:19:23.385043 | INFO | Copyright (C) 2018-2022 K4YT3X and contributors.
17:19:23.385209 | INFO | Reading input video information
17:19:23.483247 | INFO | Starting video decoder
17:19:23.490007 | INFO | Starting video encoder
17:19:23.500278 | INFO | Upscaler process 0 initiating
17:19:23.503184 | INFO | Upscaler process 1 initiating
17:19:23.506867 | INFO | Upscaler process 2 initiating
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
[..]The first error seems to be caused by your system not running with systemd so you have to manually instruct Podman to use cgroupfs instead of systemd's implementation of cgroup. The second one I'm not quite sure. Might need some more info. What's your:
Also try running vulkaninfo on your host to see if it works.
plambe
commented
2 years ago $ vulkaninfo | head
ERROR: [Loader Message] Code 0 : /usr/lib/i386-linux-gnu/libvulkan_radeon.so: wrong ELF class: ELFCLASS32
ERROR: [Loader Message] Code 0 : /usr/lib/i386-linux-gnu/libvulkan_lvp.so: wrong ELF class: ELFCLASS32
ERROR: [Loader Message] Code 0 : /usr/lib/i386-linux-gnu/libvulkan_intel.so: wrong ELF class: ELFCLASS32
WARNING: lavapipe is not a conformant vulkan implementation, testing use only.
==========
VULKANINFO
==========
Vulkan Instance Version: 1.2.162
Instance Extensions: count = 18
===============================
VK_EXT_acquire_xlib_display : extension revision 1vulkaninfo works in a docker container as well:
$ docker run --rm --gpus 1 nvidia/vulkan:1.2.133-450 vulkaninfo | head
'DISPLAY' environment variable not set... skipping surface info
error: XDG_RUNTIME_DIR not set in the environment.
==========
VULKANINFO
==========
Vulkan Instance Version: 1.2.131
Instance Extensions: count = 17
====================
VK_EXT_acquire_xlib_display : extension revision 1Driver Version: 510.47.03 CUDA Version: 11.6I have installed nvidia-container-toolkit which is same thing as nvidia-docker from my understanding. I run podman as it is shipped already in this distro, and i don't have docker.
I found out that i --gpus=all --privileged fixed my issuies with vulkaninfo in the container. I don't have vulkan on the host, but nvidia-smi output in this container is identical to host's output.
[alexander@fatserver ~]$ sudo podman run --rm --gpus=all --privileged nvidia/vulkan:1.2.133-450 vulkaninfo
'DISPLAY' environment variable not set... skipping surface info
error: XDG_RUNTIME_DIR not set in the environment.
==========
VULKANINFO
==========
Vulkan Instance Version: 1.2.131
Instance Extensions: count = 18
====================
VK_EXT_acquire_xlib_display : extension revision 1
VK_EXT_debug_report : extension revision 10
VK_EXT_debug_utils : extension revision 2
VK_EXT_direct_mode_display : extension revision 1
VK_EXT_display_surface_counter : extension revision 1
VK_KHR_device_group_creation : extension revision 1
VK_KHR_display : extension revision 23
VK_KHR_external_fence_capabilities : extension revision 1
VK_KHR_external_memory_capabilities : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2 : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2 : extension revision 1
VK_KHR_surface : extension revision 25
VK_KHR_surface_protected_capabilities : extension revision 1
VK_KHR_wayland_surface : extension revision 6
VK_KHR_xcb_surface : extension revision 6
VK_KHR_xlib_surface : extension revision 6
Layers: count = 8
=======I still have an error with running code tho:
[alexander@fatserver ~]$ sudo podman run -it --rm -v /home/alexander/vid_host/:/host/:z --privileged --gpus='all' ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i /host/video.mp4 -o /host/chad_video.mp4 -p4 upscale -h 2160 -a srmd -n3
20:09:23.134531 | INFO | Video2X 5.0.0-beta5
20:09:23.134715 | INFO | Copyright (C) 2018-2022 K4YT3X and contributors.
20:09:23.134848 | INFO | Reading input video information
20:09:23.325745 | INFO | Starting video decoder
20:09:23.331929 | INFO | Starting video encoder
20:09:23.337927 | INFO | Upscaler process 0 initiating
20:09:23.339165 | INFO | Upscaler process 1 initiating
20:09:23.340332 | INFO | Upscaler process 2 initiating
20:09:23.341506 | INFO | Upscaler process 3 initiating
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/host/video.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2mp41
creation_time : 2022-04-06T10:03:15.000000Z
encoder : Blackmagic Design DaVinci Resolve
Duration: 01:46:26.40, start: 0.000000, bitrate: 42530 kb/s
Stream #0:0(und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 42330 kb/s, 50 fps, 50 tbr, 12800 tbn, 50 tbc (default)
Metadata:
creation_time : 2022-04-06T10:03:15.000000Z
handler_name : VideoHandler
timecode : 01:00:00:00
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 196 kb/s (default)
Metadata:
creation_time : 2022-04-06T10:03:15.000000Z
handler_name : SoundHandler
Stream #0:2(eng): Data: none (tmcd / 0x64636D74)
Metadata:
creation_time : 2022-04-06T10:03:15.000000Z
handler_name : TimeCodeHandler
timecode : 01:00:00:00
Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> rawvideo (native))
Output #0, rawvideo, to 'pipe:1':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2mp41
encoder : Lavf58.29.100
Stream #0:0(und): Video: rawvideo (RGB[24] / 0x18424752), rgb24, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 2488320 kb/s, 50 fps, 50 tbn, 50 tbc (default)
Metadata:
creation_time : 2022-04-06T10:03:15.000000Z
handler_name : VideoHandler
timecode : 01:00:00:00
encoder : Lavc58.54.100 rawvideo
vkEnumeratePhysicalDevices failed -3vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
vkEnumeratePhysicalDevices failed -3
20:09:25.343381 | ERROR | process died unexpectedly
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
| | -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free software: you...
| -> <code object <module> at 0x7f7b2d6b9f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
-> <function _run_code at 0x7f7b3d0ab040>
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
| -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free software: you...
-> <code object <module> at 0x7f7b2d6b9f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
File "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 30, in <module>
sys.exit(main())
| | -> <function main at 0x7f7b2d63f040>
| -> <built-in function exit>
-> <module 'sys' (built-in)>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 612, in main
video2x.upscale(
| -> <function Video2X.upscale at 0x7f7b2d6d4ee0>
-> <video2x.video2x.Video2X object at 0x7f7b2d646040>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 412, in upscale
self._run(
| -> <function Video2X._run at 0x7f7b2d6d4e50>
-> <video2x.video2x.Video2X object at 0x7f7b2d646040>
> File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 308, in _run
raise Exception("process died unexpectedly")
Exception: process died unexpectedly
20:09:25.352940 | INFO | Stopping processor processes
20:09:25.353654 | INFO | Stopping decoder and encoder threads
20:09:25.366093 | DEBUG | Encoding queue depleted
20:09:25.382899 | DEBUG | Decoding queue depleted
20:09:25.837951 | INFO | Decoder thread exiting
20:09:25.839251 | INFO | Encoder thread exiting
20:09:25.839977 | ERROR | process died unexpectedly
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
| | -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free software: you...
| -> <code object <module> at 0x7f7b2d6b9f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
-> <function _run_code at 0x7f7b3d0ab040>
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
| -> {'__name__': '__main__', '__doc__': '\nCopyright (C) 2018-2022 K4YT3X and contributors.\n\nThis program is free software: you...
-> <code object <module> at 0x7f7b2d6b9f50, file "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 3>
File "/usr/local/lib/python3.8/dist-packages/video2x/__main__.py", line 30, in <module>
sys.exit(main())
| | -> <function main at 0x7f7b2d63f040>
| -> <built-in function exit>
-> <module 'sys' (built-in)>
> File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 612, in main
video2x.upscale(
| -> <function Video2X.upscale at 0x7f7b2d6d4ee0>
-> <video2x.video2x.Video2X object at 0x7f7b2d646040>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 412, in upscale
self._run(
| -> <function Video2X._run at 0x7f7b2d6d4e50>
-> <video2x.video2x.Video2X object at 0x7f7b2d646040>
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 375, in _run
raise exception[0]
-> [Exception('process died unexpectedly')]
File "/usr/local/lib/python3.8/dist-packages/video2x/video2x.py", line 308, in _run
raise Exception("process died unexpectedly")
Exception: process died unexpectedlyI don't really see why it would fail. Can you try adding --device=/dev/dri? Maybe it's cgroup permissions?
AChmykhov
commented
2 years ago It doesn't work with --device=/dev/dri, --gpus='all', --gpus='all,"capabilities=compute,utility,graphics,display"', any combination of them or without any of them :(
Theelx
commented
2 years ago I had this same issue, and I fixed it using
sudo docker run -it --rm -v /home/theel/Videos/:/host --gpus='all,"capabilities=compute,utility,graphics,display"' --env DISPLAY:$DISPLAY ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i input.mp4 -o output2x.mp4 -p1 upscale -h 720 -a waifu2x -n3Specifically, I added --gpus='all,"capabilities=compute,utility,graphics,display"' --env DISPLAY:$DISPLAY as stated here. Interestingly, this worked when running with docker, but not when I ran with podman 3.4.2.
AChmykhov
commented
2 years ago Small update: according to podman run documentation there is no such an option as --gpus : https://docs.podman.io/en/latest/markdown/podman-run.1.html
Try adding --privileged , it helped me somewhere (like running nvidia-smi or vulkaninfo in container).
Also you need :z flag after path in volume for podman. In your case it will be -v /home/theel/Videos/:/host:z .
plambe
commented
2 years ago I tried @Theelx's @AChmykhov's methods and there was no difference for me
AChmykhov
commented
2 years ago Hey, i tried to figure out why vulkan is not working in this specific container, set entrypoint as /bin/bash and run vulkaninfo. Here is error it's returning:
[alexander@fatserver ~]$ sudo podman run -it --rm -v /home/alexander/vid_host/:/host/:z --privileged --security-opt=label=disable --hooks-dir=/usr/share/containers/oci/hooks.d/ --entrypoint=/bin/bash ghcr.io/k4yt3x/video2x:5.0.0-beta5
root@1196233542f3:/host# vulkaninfo
ERROR: [Loader Message] Code 0 : loader_scanned_icd_add: Could not get 'vkCreateInstance' via 'vk_icdGetInstanceProcAddr' for ICD libGLX_nvidia.so.0
'DISPLAY' environment variable not set... skipping surface info
error: XDG_RUNTIME_DIR not set in the environment.
/root/sdk-build/1.2.131.2/source/Vulkan-Tools/vulkaninfo/vulkaninfo.h:477: failed with ERROR_INITIALIZATION_FAILEDIn nvidia/vulkan:1.2.133-450 variable XDG_RUNTIME_DIR is set to /run/user/1000.
AChmykhov
commented
2 years ago Okay i probably don't understand how podman work, but i have different results for to commands that should be same: (added host result just as reference)
[alexander@fatserver ~]$ sudo podman run -it --rm --privileged nvidia/vulkan:1.2.133-450 echo $XDG_RUNTIME_DIR
/run/user/1000
[alexander@fatserver ~]$ sudo podman run -it --rm --privileged nvidia/vulkan:1.2.133-450
root@d3dcce12c9c8:/# echo $XDG_RUNTIME_DIR
root@d3dcce12c9c8:/# exit
[alexander@fatserver ~]$ echo $XDG_RUNTIME_DIR
/run/user/1000Also if nvidia/vulkan:1.2.133-450 doesn't have XDG_RUNTIME_DIR set, why vulkaninfo works here, but not in video2x container?
AChmykhov
commented
2 years ago Okay, so this vulkaninfo error is connected with nvidia driver error. nvidia-smi is working on host and in nvidia/vulkan:1.2.133-450 container, but in video2x container it returns Failed to initialize NVML: Driver/library version mismatch.
plambe
commented
2 years ago While I was actively trying to find a solution to this, I noticed a mismatch between my OS driver version and this container's driver version. I'll try to build the video2x container image myself with my OS driver version when I get home.
AChmykhov
commented
2 years ago I tried to upgrade container's driver inside container, but encountered some issues due to minimized system and updated nvidia's keyring. So i followed this tutorial for keyring, and also updated base container's version to docker.io/nvidia/vulkan:1.2.170-470 just in case. After apt update and apt upgrade (I used interactive mode on container, but there probably is -y option for this) it all started working (I didn't let it fully run due to need to run heavy load tasks in specific time of day, but it went past points, when it returned errors). My current Dockerfile:
# Name: Video2X Dockerfile
# Creator: K4YT3X
# Date Created: February 3, 2022
# Last Modified: March 28, 2022
# stage 1: build the python components into wheels
FROM docker.io/nvidia/vulkan:1.2.170-470 AS builder
ENV DEBIAN_FRONTEND=noninteractive
COPY . /video2x
WORKDIR /video2x
RUN rm /etc/apt/sources.list.d/cuda.list \
&& dpkg -i cuda-keyring_1.0-1_all.deb
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
python3.8 python3-pip python3-opencv python3-pil \
python3.8-dev libvulkan-dev glslang-dev glslang-tools \
build-essential swig \
&& pip wheel -w /wheels wheel pdm-pep517 .
# stage 2: install wheels into the final image
FROM docker.io/nvidia/vulkan:1.2.170-470
LABEL maintainer="K4YT3X <i@k4yt3x.com>" \
org.opencontainers.image.source="https://github.com/k4yt3x/video2x" \
org.opencontainers.image.description="A lossless video/GIF/image upscaler"
ENV DEBIAN_FRONTEND=noninteractive
ENV VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nvidia_icd.json\
:/usr/share/vulkan/icd.d/radeon_icd.x86_64.json\
:/usr/share/vulkan/icd.d/intel_icd.x86_64.json
COPY --from=builder /var/lib/apt/lists* /var/lib/apt/lists/
COPY --from=builder /wheels /wheels
COPY . /video2x
WORKDIR /video2x
RUN rm /etc/apt/sources.list.d/cuda.list \
&& dpkg -i cuda-keyring_1.0-1_all.deb
RUN apt-get update
RUN apt-get install -y --no-install-recommends \
python3-pip python3.8-dev \
python3-opencv python3-pil \
mesa-vulkan-drivers cuda-drivers ffmpeg \
&& pip install --no-cache-dir --no-index -f /wheels . \
&& apt-get clean \
&& rm -rf /wheels /video2x /var/lib/apt/lists/*
WORKDIR /host
ENTRYPOINT ["/usr/bin/python3.8", "-m", "video2x"]Then i run it like this:
[alexander@fatserver video2x]$ sudo podman run -it --rm -v /home/alexander/vid_host/:/host/:z --privileged --entrypoint=/bin/bash --gpus=all --network=host localhost/videohell3
root@fatserver:/host# apt update
[ . . . ]
root@fatserver:/host# apt upgrade
[ . . . ]
root@fatserver:/host# /usr/bin/python3.8 -m video2x -i /host/video.mp4 -o /host/chad_video.mp4 -p4 upscale -h 2160 -a srmd -n3
[ . . . ]Thanks for sharing. This is great information for others with the same problem. It seems different users on different distros have various different issues that I can't cover just by testing on my machines. I think it's worth gathering all these solutions somewhere into a Q&A so people can refer to it as a handbook when they encounter the same issues (perhaps in the wiki).
plambe
commented
2 years ago I finally got it to work for me. In short, I had to both forward some devices from /dev (similar to podman run -it --rm --device /dev/nvidia0 --device /dev/nvidia-uvm --device /dev/nvidia-uvm-tools --device /dev/nvidiactl --privileged --entrypoint=/bin/bash -v $(pwd):/host ghcr.io/k4yt3x/video2x:5.0.0-beta5) and install the exact same version of all nvidia stuff as on my host OS. At the moment I'm trying to patch the Dockerfile so that it builds an image properly for my host OS.
For this installation to work, the nvidia key has to be updated as explained by @AChmykhov in this comment https://github.com/k4yt3x/video2x/issues/628#issuecomment-1114782673. Here's an example of how I install the proper nvidia stuff - version 460.91.03:
apt install -y --allow-downgrades \
cuda-drivers-460=460.91.03-1 nvidia-driver-460=460.91.03-0ubuntu1 \
libnvidia-gl-460=460.91.03-0ubuntu1 nvidia-dkms-460=460.91.03-0ubuntu1 \
nvidia-kernel-source-460=460.91.03-0ubuntu1 libnvidia-compute-460=460.91.03-0ubuntu1 \
libnvidia-extra-460=460.91.03-0ubuntu1 nvidia-compute-utils-460=460.91.03-0ubuntu1 \
libnvidia-decode-460=460.91.03-0ubuntu1 libnvidia-encode-460=460.91.03-0ubuntu1 \
nvidia-utils-460=460.91.03-0ubuntu1 xserver-xorg-video-nvidia-460=460.91.03-0ubuntu1 \
libnvidia-cfg1-460=460.91.03-0ubuntu1 libnvidia-ifr1-460=460.91.03-0ubuntu1 \
libnvidia-fbc1-460=460.91.03-0ubuntu1 libnvidia-common-460=460.91.03-0ubuntu1 \
nvidia-kernel-common-460=460.91.03-0ubuntu1I'll try to document this better for other people using rarer linux flavours with outdated nvidia drivers and libs.
plambe
commented
2 years ago I documented a minimal set of steps to fix the issue I had (with the host OS nvidia driver version) here: https://github.com/k4yt3x/video2x/wiki/Container#troubleshooting-issues-with-nvidia-gpus-with-old-drivers-in-the-host-operating-system There's a patched version of the original Dockerfile embedded in the wiki.
I thought of creating a pull request to add this as Dockerfile.oldnvidiadriver as I tried to generalize it somewhat, but decided against that because my exact combination of linux flavour, nvidia driver, etc. is probably quite rare.
k4yt3x
commented
2 years ago @plambe Thanks for the contribution. I fixed some markdown formatting errors and added a credit the section.
ilbuonmarcio
commented
2 years ago I had this same issue, and I fixed it using
sudo docker run -it --rm -v /home/theel/Videos/:/host --gpus='all,"capabilities=compute,utility,graphics,display"' --env DISPLAY:$DISPLAY ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i input.mp4 -o output2x.mp4 -p1 upscale -h 720 -a waifu2x -n3Specifically, I added
--gpus='all,"capabilities=compute,utility,graphics,display"' --env DISPLAY:$DISPLAYas stated here. Interestingly, this worked when running with docker, but not when I ran with podman 3.4.2.
This one fixed my error in Arch Linux using an RTX 3080 Ti with nvidia-container-toolkit and docker with the following command:
sudo docker run -it --rm --privileged --gpus='all,"capabilities=compute,utility,graphics,display"' --env DISPLAY:$DISPLAY -v $PWD:/host ghcr.io/k4yt3x/video2x:5.0.0-beta4-cuda-patched -i input.mp4 -o output.mp4 -p1 upscale -h 720 -a waifu2x -n3Otherwise it shows errors as shown above by others!
k4yt3x
commented
2 years ago I updated the Container docs again. It should work properly right now following the instructions given. I'll close this issue because it's gotten too long. Please open new issues if you still can't get it to work. Feel free to keep appending suggestions here though.
johnnybaloney
commented
1 year ago For me solving the vkEnumeratePhysicalDevices failed -3 problem running video2x:5.0.0-beta5 on Ubuntu 18 turned out to be as simple as bumping the driver version in the proprietary driver settings from nvidia 470 to 530 (and then regenerating the cdi spec etc.).
Trevoke
commented
7 months ago I'm in Ubuntu 22 and I use an AMD Radeon RX 5700 XT.
The documentation for running from a container failed for me with the vkEnumeratePhysicalDevices failed -3 error, however, a comment from above helped me get this running.
I have two devices for my card under /dev/dri/ and I had to give the container CLI --device=/dev/dri/card0 --device=/dev/dri/renderD128.
dagguh
commented
6 months ago I have two devices for my card under
/dev/dri/and I had to give the container CLI--device=/dev/dri/card0 --device=/dev/dri/renderD128.
This works!
I cannot get the Docker image to work. The syntax given is
docker run -it --rm -v $PWD:/host ghcr.io/k4yt3x/video2x:5.0.0-beta5 -i input.mp4 -o output.mp4 -p6 upscale -h 720 -a waifu2x -n3but this fails:Upon running --help to see if I can mess around and get things to work,
Almost none of the flags given in the documentation actually exist, and ones that do are wrong. For example
-hprints the help message, it does not set height (as it does in 4.3.1).