search

voidmainvoid95 / gst-nvmaxine

Gstreamer plugin that allows use of NVIDIA Maxine SDK in a generic pipeline.
GNU Lesser General Public License v2.1
32 stars 1 forks source link

(erroneous?) GPU not supported on datacenter RTX A4000 #6

Closed flockman66 closed 5 months ago

flockman66 commented 5 months ago

The basic pipeline seems to fail with GPU not supported error when testing through the provided docker image

NVIDIA-SMI root@582fca7de7cd:/shared/gst-nvmaxine/build# nvidia-smi Sun May 19 16:08:26 2024
+---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.161.08 Driver Version: 535.161.08 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA RTX 4000 SFF Ada ... Off | 00000000:01:00.0 Off | Off | | 30% 32C P8 5W / 70W | 3MiB / 20475MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | No running processes found | +---------------------------------------------------------------------------------------+

ERROR

t@582fca7de7cd:/shared/gst-nvmaxine/build# gst-launch-1.0 filesrc location="/shared/Big_Buck_Bunny_720_10s_5MB.mp4" ! qtdemux ! avdec_h264 ! queue ! nvmaxinevideofx effect=SuperRes modeldir="/usr/local/VideoFX/lib/models" ! queue ! x264enc ! qtmux ! filesink location="example_video_output.mp4" Setting pipeline to PAUSED ... 0:00:00.056353274 331 0x6034deb18e00 WARN basesrc gstbasesrc.c:3600:gst_base_src_start_complete: pad not activated yet Pipeline is PREROLLING ... 0:00:00.056479573 331 0x6034ddfa2e40 WARN qtdemux qtdemux_types.c:239:qtdemux_type_get: unknown QuickTime node type pasp 0:00:00.056514235 331 0x6034ddfa2e40 WARN qtdemux qtdemux.c:3250:qtdemux_parse_trex: failed to find fragment defaults for stream 1 Redistribute latency... 0:00:00.131230869 331 0x6034ddfa2de0 ERROR nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported 0:00:00.132698684 331 0x6034ddfa2de0 WARN basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 0:00:00.133006697 331 0x6034ddfa2de0 ERROR nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported 0:00:00.134440667 331 0x6034ddfa2de0 WARN basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 0:00:00.134448681 331 0x6034ddfa2de0 WARN GST_PADS gstpad.c:4231:gst_pad_peer_query: could not send sticky events Redistribute latency... 0:00:00.135148417 331 0x6034ddfa2de0 ERROR nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported 0:00:00.136604369 331 0x6034ddfa2de0 WARN basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 0:00:00.136922718 331 0x6034ddfa2de0 ERROR nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported 0:00:00.138371278 331 0x6034ddfa2de0 WARN basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 0:00:00.138694887 331 0x6034ddfa2de0 ERROR nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported 0:00:00.140130318 331 0x6034ddfa2de0 WARN basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 0:00:00.140454914 331 0x6034ddfa2de0 ERROR nvmaxinevideofx gstnvmaxinevideofx.c:399:gst_nv_maxine_videofx_set_caps: The GPU is not supported 0:00:00.141893895 331 0x6034ddfa2de0 WARN basetransform gstbasetransform.c:1370:gst_base_transform_setcaps: FAILED to configure incaps video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 and outcaps video/x-raw, format=(string)I420, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1 0:00:00.141919942 331 0x6034ddfa2e40 WARN qtdemux qtdemux.c:6619:gst_qtdemux_loop: error: Internal data stream error. 0:00:00.141930157 331 0x6034ddfa2e40 WARN qtdemux qtdemux.c:6619:gst_qtdemux_loop: error: streaming stopped, reason not-negotiated (-4) ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux0: Internal data stream error. Additional debug info: qtdemux.c(6619): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstQTDemux:qtdemux0: streaming stopped, reason not-negotiated (-4) ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... Freeing pipeline ...

flockman66 commented 5 months ago

i've also tried this on my local machines 4080(mobile), with the same results, initially i thought it was due to it being a consumer part, so i tried it on our cloud gpu, but i get the exact same result

voidmainvoid95 commented 5 months ago

Hi, as stated here https://docs.nvidia.com/deeplearning/maxine/vfx-sdk-system-guide/index.html#vfx-sdk-system-guide-linux-intro

The Video Effects SDK is compatible with GPUs that are based on NVIDIA Turing™, NVIDIA Volta™, or the NVIDIA Ampere™ architecture.

Both RTX 4000 and RTX4080 are based on Ada Lovelace, so they are not supported. Watch out that you are not using an RTX A4000 (which is based on Ampere architecture) but an RTX 4000 SFF Ada, as nvidia-smi reports.

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.08 Driver Version: 535.161.08 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX 4000 SFF Ada ... Off | 00000000:01:00.0 Off | Off |
| 30% 32C P8 5W / 70W | 3MiB / 20475MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| No running processes found |
+---------------------------------------------------------------------------------------+

I've a 2080ti and plugins runs fine using the provided Docker image. However the plugins fails to run using a ubuntu 22.04 image configured with the same dependencies and the same libraries version, saying that no model can run on my system.

Unfortunately, Nvidia is used to limiting libraries to certain distribution, versions of hardware and/or software. :disappointed:

flockman66 commented 5 months ago

Ah crystal, thanks for replying, im new to nvidia ML, mostly used rocm before (came over to team green because its supposed to "just work"), feels strange that itd be better for me to get a 2000/3000 series though.

flockman66 commented 5 months ago

closing.

flockman66 commented 5 months ago

bafflingly it seems ADA is supported by the VFX SDK but only on windows , really hope thats just temporary, and not some bone-headed contractual idiocy between M$ and nvidia

voidmainvoid95 commented 5 months ago

Ah crystal, thanks for replying, im new to nvidia ML, mostly used rocm before (came over to team green because its supposed to "just work"), feels strange that itd be better for me to get a 2000/3000 series though.

In most of the cases your sentence is correct. A lot of ML frameworks and cuda based projects work on nvidia platform without any sort of problem. However a lot of them are open source projects, so you can always write a patch to make them run on hardware that is not officially supported. Maxine is a proprietary framework shipped in binary form, so if nvidia doesn't provided a compatible build for your system, you are screwed. Sometimes you can make a workaround (see https://github.com/voidmainvoid95/gst-nvmaxine/issues/4#issuecomment-2119273355), sometimes not.

I'm sure that the next linux release will support your GPU as well, but I don't now when Nvidia will release it.

bafflingly it seems ADA is supported by the VFX SDK but only on windows , really hope thats just temporary, and not some bone-headed contractual idiocy between M$ and nvidia

The real reason, probably, is that on windows maxine sdk is shipped as a part of Nvidia Broadcast App (https://nvidia.com/it-it/geforce/broadcasting/broadcast-app/), so they need a wider support.

flockman66 commented 5 months ago

Good to know, we're doing a de-noising and up-scaling pipeline for real-time video so this pipeline is really intriguing to us/me, plenty of open source upscaling/denoising that uses cuda, but they are all offline-compute and very slow, i'll try and get a 3080 or data-center equivalent i really would like to try this out.

Thanks for the explanation :)

rocco-haro commented 1 month ago

@flockman66 @voidmainvoid95

Hey folks. I heard back from Nvidia. Supposedly it does support ADA on ubuntu 20 and 22. I think nvidia is notorious for not updating their documentation.

These latest releases are also blocked behind a "paywall"

From Nvidia support: 

AR: https://registry.ngc.nvidia.com/orgs/ea-maxine/teams/ar-linux-sdk/collections/maxine_linux_ar_sdk_ea_collection

Video: https://registry.ngc.nvidia.com/orgs/ea-maxine/teams/vfx-linux-sdk/collections/maxine_linux_video_effects_sdk_collection

AFX: https://registry.ngc.nvidia.com/orgs/ea-maxine/teams/afx-linux-sdk/collections/maxine_linux_audio_effects_sdk_collection

Currently we support Ubuntu 18, 20 and 22 across all three SDK’s but we are considering dropping support for Ubuntu 18 from next release onwards
voidmainvoid95 commented 1 month ago

Hey @rocco-haro , thanks for the update!

Unfortunately, this is typical behavior from Nvidia :rage:.

The fact that these latest releases are blocked behind a "paywall" makes it difficult for me to continue supporting the plugin. Even though I don't need the redistributable binary, I still require access to the libraries for development, which I can't get without a company license or affiliation.