search

mpv-player / mpv

🎥 Command line video player
https://mpv.io
Other
27.77k stars 2.86k forks source link

Vulkan Video Decoding: Usage Guide and FAQ #11739

Closed philipl closed 4 months ago

philipl commented 1 year ago

Vulkan Video Decoding: Usage Guide and FAQ

On the 28th of May, we reached the significant milestone of finally merging all the required functionality into ffmpeg, libplacebo, and mpv to do end-to-end Vulkan video decoding and presentation. Although the functionality is now all there, we still have a complex landscape in terms of what is supported on what hardware and with what drivers. This document attempts to lay out all the requirements and limitations, so you have a chance of successfully using the feature.

Why should I even care about Vulkan video decoding?

It's a fair question. Right now, the actual functionality you gain access to is not terribly different from what you would get with existing video decoding APIs that ffmpeg and mpv already support. However, Vulkan video decoding has the potential to be a credible cross-vendor, cross-platform API that is well supported, and can work efficiently with Vulkan based filtering and post-processing. In the short term, there will be rough edges, and only a narrow set of supported codecs (only H.264 and H.265 are standardised today, with AV1 still a work in progress), but in the future, it will hopefully be the most performant and capable way to do hardware decoding and processing.

Software Requirements

We have finally reached the point where all the required components have made official releases. Ensure you are running with the following releases or newer:

These requirements mean you will likely have to compile everything for yourself (especially on Linux), but Shinchiro's windows builds appear to be functional.

Hardware and Driver Requirements

Usage

The basic command line arguments:

mpv --vo=gpu-next --gpu-api=vulkan --hwdec=vulkan

Depending on your system configuration, you may additionally need to force the right gpu-context:

Capabilities

kasper93 commented 1 year ago

AMD released new preview driver for Windows with VK_KHR_videodecode* support among other things: AMD Software: Adrenalin Edition 23.10.01.41 for Additional Vulkan Extensions Release Notes

I tried on 5700 XT (RDNA) and it basically causes device reset with both h264 and hevc. YMMV with different hardware. AMD Driver: 23.10.01.41 mpv: https://github.com/mpv-player/mpv/commit/e7902eb83dc68dcb0adb99568d72bd3f9a4e4550 FFmpeg: https://github.com/FFmpeg/FFmpeg/commit/6ff95758e6bf47004f69c56d7547a971c4782b21 libplacebo: https://github.com/haasn/libplacebo/commit/8ceb7a544b70aff7e432cbb9708666115a6cb985

Some logs:

vulkan-copy ``` [vd] Requesting pixfmt 'vulkan' from decoder. [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST [ffmpeg/video] h264: hardware accelerator failed to decode picture [vd] Error while decoding frame (hardware decoding)! [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST [ffmpeg/video] h264: get_buffer() failed [ffmpeg/video] h264: decode_slice_header error [ffmpeg/video] h264: no frame! [vd] Error while decoding frame (hardware decoding)! ```
vulkan (w/ validation) ``` [vo/gpu-next/libplacebo] vk Validation Error: [ VUID-VkImageCreateInfo-pNext-06811 ] Object 0: handle = 0x12a64bf16f60, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x30f4ac70 | vkCreateImage: image creation parameters (flags: 0x00000508, format: VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, imageType: VK_IMAGE_TYPE_2D, tiling: VK_IMAGE_TILING_OPTIMAL) are not supported by any of the supported video format properties for the video profiles specified in the VkVideoProfileListInfoKHR structure included in the pCreateInfo->pNext chain, as reported by vkGetPhysicalDeviceVideoFormatPropertiesKHR for the same video profiles and the image usage flags specified in pCreateInfo->usage (0x00000405) The Vulkan spec states: If the pNext chain includes a VkVideoProfileListInfoKHR structure with profileCount greater than 0, then supportedVideoFormat must be VK_TRUE (https://vulkan.lunarg.com/doc/view/1.3.250.0/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-06811) ```
vulkan (w/o validation) ``` [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST [ffmpeg/video] h264: hardware accelerator failed to decode picture [vd] Error while decoding frame (hardware decoding)! [statusline] AV: 00:00:00.000 / 00:54:22.743 (0%) A-V: 0.000 DS: 2.000/0 [vo/gpu-next/libplacebo] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:358) [vo/gpu-next/libplacebo] vkQueueSubmit2: VK_ERROR_DEVICE_LOST (../src/vulkan/command.c:358) [vo/gpu-next/libplacebo] Failed holding swapchain image for presentation [vo/gpu-next] Failed presenting frame! [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST [ffmpeg/video] h264: get_buffer() failed [ffmpeg/video] h264: decode_slice_header error [ffmpeg/video] h264: no frame! [vd] Error while decoding frame (hardware decoding)! [cplayer] Audio device underrun detected. [cplayer] restarting audio after underrun [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Failed to create semaphore: VK_ERROR_DEVICE_LOST [ffmpeg/video] h264: get_buffer() failed [ffmpeg/video] h264: decode_slice_header error [ffmpeg/video] h264: no frame! [vd] Error while decoding frame (hardware decoding)! [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST ```
Jules-A commented 1 year ago

AMD released new preview driver with VK_KHR_videodecode* among other things: AMD Software: Adrenalin Edition 23.10.01.41 for Additional Vulkan Extensions Release Notes

Just tried it... EDIT: I'm blind, still no support for RX570, it was falling back to software... Still doesn't have the extension.

cyanreg commented 1 year ago

Could you/anyone else with the driver post the log of --msg-level=ffmpeg=debug?

kasper93 commented 1 year ago

There is not much there really, it just freeze on queue submit, reset device and throws error.

vulkan 1080p h.264 ``` [ffmpeg] AVHWDeviceContext: Using device: AMD Radeon RX 5700 XT [ffmpeg] AVHWDeviceContext: Alignments: [ffmpeg] AVHWDeviceContext: optimalBufferCopyRowPitchAlignment: 1 [ffmpeg] AVHWDeviceContext: minMemoryMapAlignment: 64 [ffmpeg] AVHWDeviceContext: nonCoherentAtomSize: 128 [ffmpeg] AVHWDeviceContext: minImportedHostPointerAlignment: 4096 [ffmpeg] AVHWDeviceContext: Using queue family 0 (queues: 1) for graphics encode [ffmpeg] AVHWDeviceContext: Using queue family 2 (queues: 1) for transfers [ffmpeg] AVHWDeviceContext: Using queue family 1 (queues: 1) for compute [ffmpeg] AVHWDeviceContext: Using queue family 4 (queues: 1) for decode [ffmpeg/video] h264: Decoder capabilities for h264 profile "High": [ffmpeg/video] h264: Maximum level: 0 (stream 40) [ffmpeg/video] h264: Width: from 16 to 4096 [ffmpeg/video] h264: Height: from 16 to 4096 [ffmpeg/video] h264: Width alignment: 16 [ffmpeg/video] h264: Height alignment: 16 [ffmpeg/video] h264: Bitstream offset alignment: 4096 [ffmpeg/video] h264: Bitstream size alignment: 4096 [ffmpeg/video] h264: Maximum references: 17 [ffmpeg/video] h264: Maximum active references: 17 [ffmpeg/video] h264: Codec header version: 0.0.0 (driver), 1.0.0 (compiled) [ffmpeg/video] h264: Decode modes: dedicated_dpb [ffmpeg/video] h264: Capability flags: none [ffmpeg/video] h264: Chosen frame pixfmt: nv12 (Vulkan ID: 1000156003) F:/dev/ffmpeg/libavutil/vulkan.c:406:55: runtime error: applying zero offset to null pointer #0 0x7ff8173295b2 in ff_vk_exec_pool_init F:/dev/ffmpeg/libavutil/vulkan.c:406:55 #1 0x7ff816b4c702 in vulkan_frames_init F:/dev/ffmpeg/libavutil/hwcontext_vulkan.c:2326:11 #2 0x7ff816af16a2 in av_hwframe_ctx_init F:/dev/ffmpeg/libavutil/hwcontext.c:364:15 #3 0x7ff651974d8f in init_generic_hwaccel F:/dev/mpv/build/../video/decode/vd_lavc.c:889:13 #4 0x7ff651974d8f in get_format_hwdec F:/dev/mpv/build/../video/decode/vd_lavc.c:934:17 #5 0x7fffe26561ad in ff_get_format F:/dev/ffmpeg/libavcodec/decode.c:1198:23 #6 0x7fffe3b1934a in get_pixel_format F:/dev/ffmpeg/libavcodec/h264_slice.c:908:12 #7 0x7fffe3affdd5 in h264_init_ps F:/dev/ffmpeg/libavcodec/h264_slice.c:1147:20 #8 0x7fffe3ac2d30 in h264_field_start F:/dev/ffmpeg/libavcodec/h264_slice.c:1398:11 #9 0x7fffe3ab5480 in ff_h264_queue_decode_slice F:/dev/ffmpeg/libavcodec/h264_slice.c:2135:15 #10 0x7fffe3bb5ecd in decode_nal_units F:/dev/ffmpeg/libavcodec/h264dec.c:641:24 #11 0x7fffe3bace45 in h264_decode_frame F:/dev/ffmpeg/libavcodec/h264dec.c:1015:17 #12 0x7fffe2669344 in decode_simple_internal F:/dev/ffmpeg/libavcodec/decode.c:287:15 #13 0x7fffe266794b in decode_simple_receive_frame F:/dev/ffmpeg/libavcodec/decode.c:533:15 #14 0x7fffe2646609 in decode_receive_frame_internal F:/dev/ffmpeg/libavcodec/decode.c:553:15 #15 0x7fffe2645f9b in avcodec_send_packet F:/dev/ffmpeg/libavcodec/decode.c:641:15 #16 0x7ff65197161b in init_avctx F:/dev/mpv/build/../video/decode/vd_lavc.c:777:9 #17 0x7ff6519690ca in reinit F:/dev/mpv/build/../video/decode/vd_lavc.c:612:5 #18 0x7ff651962cc4 in create.llvm.14630122091301803147 F:/dev/mpv/build/../video/decode/vd_lavc.c:1362:5 #19 0x7ff65167d2c0 in reinit_decoder F:/dev/mpv/build/../filters/f_decoder_wrapper.c:450:22 #20 0x7ff65167d2c0 in mp_decoder_wrapper_reinit F:/dev/mpv/build/../filters/f_decoder_wrapper.c:478:16 #21 0x7ff6518af19d in init_video_decoder F:/dev/mpv/build/../player/video.c:188:10 #22 0x7ff6518afd87 in reinit_video_chain_src F:/dev/mpv/build/../player/video.c:257:14 #23 0x7ff651868b35 in play_current_file F:/dev/mpv/build/../player/loadfile.c:1706:5 #24 0x7ff6518629c9 in mp_play_files F:/dev/mpv/build/../player/loadfile.c:1965:13 #25 0x7ff65187a88c in mpv_main F:/dev/mpv/build/../player/main.c:438:9 #26 0x7ff651af9b19 in main F:/dev/mpv/build/../osdep/main-fn-win.c:74:15 #27 0x7ff651491314 in __tmainCRTStartup C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:267:15 #28 0x7ff651491365 in .l_start C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:188:9 #29 0x7ff871e07613 (C:\Windows\System32\KERNEL32.DLL+0x180017613) #30 0x7ff8723e26b0 (C:\Windows\SYSTEM32\ntdll.dll+0x1800526b0) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior F:/dev/ffmpeg/libavutil/vulkan.c:406:55 in F:/dev/ffmpeg/libavutil/vulkan.c:406:55: runtime error: applying zero offset to null pointer #0 0x7fffe88e90e2 in ff_vk_exec_pool_init F:/dev/ffmpeg/libavutil/vulkan.c:406:55 #1 0x7fffe8955704 in ff_vk_decode_init F:/dev/ffmpeg/libavcodec/vulkan_decode.c:1131:11 #2 0x7fffe265912e in hwaccel_init F:/dev/ffmpeg/libavcodec/decode.c:1143:15 #3 0x7fffe2657dbb in ff_get_format F:/dev/ffmpeg/libavcodec/decode.c:1281:19 #4 0x7fffe3b1934a in get_pixel_format F:/dev/ffmpeg/libavcodec/h264_slice.c:908:12 #5 0x7fffe3affdd5 in h264_init_ps F:/dev/ffmpeg/libavcodec/h264_slice.c:1147:20 #6 0x7fffe3ac2d30 in h264_field_start F:/dev/ffmpeg/libavcodec/h264_slice.c:1398:11 #7 0x7fffe3ab5480 in ff_h264_queue_decode_slice F:/dev/ffmpeg/libavcodec/h264_slice.c:2135:15 #8 0x7fffe3bb5ecd in decode_nal_units F:/dev/ffmpeg/libavcodec/h264dec.c:641:24 #9 0x7fffe3bace45 in h264_decode_frame F:/dev/ffmpeg/libavcodec/h264dec.c:1015:17 #10 0x7fffe2669344 in decode_simple_internal F:/dev/ffmpeg/libavcodec/decode.c:287:15 #11 0x7fffe266794b in decode_simple_receive_frame F:/dev/ffmpeg/libavcodec/decode.c:533:15 #12 0x7fffe2646609 in decode_receive_frame_internal F:/dev/ffmpeg/libavcodec/decode.c:553:15 #13 0x7fffe2645f9b in avcodec_send_packet F:/dev/ffmpeg/libavcodec/decode.c:641:15 #14 0x7ff65197161b in init_avctx F:/dev/mpv/build/../video/decode/vd_lavc.c:777:9 #15 0x7ff6519690ca in reinit F:/dev/mpv/build/../video/decode/vd_lavc.c:612:5 #16 0x7ff651962cc4 in create.llvm.14630122091301803147 F:/dev/mpv/build/../video/decode/vd_lavc.c:1362:5 #17 0x7ff65167d2c0 in reinit_decoder F:/dev/mpv/build/../filters/f_decoder_wrapper.c:450:22 #18 0x7ff65167d2c0 in mp_decoder_wrapper_reinit F:/dev/mpv/build/../filters/f_decoder_wrapper.c:478:16 #19 0x7ff6518af19d in init_video_decoder F:/dev/mpv/build/../player/video.c:188:10 #20 0x7ff6518afd87 in reinit_video_chain_src F:/dev/mpv/build/../player/video.c:257:14 #21 0x7ff651868b35 in play_current_file F:/dev/mpv/build/../player/loadfile.c:1706:5 #22 0x7ff6518629c9 in mp_play_files F:/dev/mpv/build/../player/loadfile.c:1965:13 #23 0x7ff65187a88c in mpv_main F:/dev/mpv/build/../player/main.c:438:9 #24 0x7ff651af9b19 in main F:/dev/mpv/build/../osdep/main-fn-win.c:74:15 #25 0x7ff651491314 in __tmainCRTStartup C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:267:15 #26 0x7ff651491365 in .l_start C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:188:9 #27 0x7ff871e07613 (C:\Windows\System32\KERNEL32.DLL+0x180017613) #28 0x7ff8723e26b0 (C:\Windows\SYSTEM32\ntdll.dll+0x1800526b0) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior F:/dev/ffmpeg/libavutil/vulkan.c:406:55 in [ffmpeg/video] h264: Allocating 226492416 bytes in bind index 0 for video session [ffmpeg/video] h264: Vulkan decoder initialization sucessful [ffmpeg/video] h264: Reinit context to 1920x1088, pix_fmt: vulkan [ffmpeg/video] h264: Decoding frame, 430 bytes, 1 slices [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST ```
vulkan 2160p h.265 ``` [ffmpeg] AVHWDeviceContext: Using device: AMD Radeon RX 5700 XT [ffmpeg] AVHWDeviceContext: Alignments: [ffmpeg] AVHWDeviceContext: optimalBufferCopyRowPitchAlignment: 1 [ffmpeg] AVHWDeviceContext: minMemoryMapAlignment: 64 [ffmpeg] AVHWDeviceContext: nonCoherentAtomSize: 128 [ffmpeg] AVHWDeviceContext: minImportedHostPointerAlignment: 4096 [ffmpeg] AVHWDeviceContext: Using queue family 0 (queues: 1) for graphics encode [ffmpeg] AVHWDeviceContext: Using queue family 2 (queues: 1) for transfers [ffmpeg] AVHWDeviceContext: Using queue family 1 (queues: 1) for compute [ffmpeg] AVHWDeviceContext: Using queue family 4 (queues: 1) for decode [ffmpeg/video] hevc: Decoder capabilities for hevc profile "Main 10": [ffmpeg/video] hevc: Maximum level: 0 (stream 150) [ffmpeg/video] hevc: Width: from 16 to 8192 [ffmpeg/video] hevc: Height: from 16 to 4352 [ffmpeg/video] hevc: Width alignment: 16 [ffmpeg/video] hevc: Height alignment: 16 [ffmpeg/video] hevc: Bitstream offset alignment: 4096 [ffmpeg/video] hevc: Bitstream size alignment: 4096 [ffmpeg/video] hevc: Maximum references: 17 [ffmpeg/video] hevc: Maximum active references: 17 [ffmpeg/video] hevc: Codec header version: 0.0.0 (driver), 1.0.0 (compiled) [ffmpeg/video] hevc: Decode modes: dedicated_dpb [ffmpeg/video] hevc: Capability flags: none [ffmpeg/video] hevc: Chosen frame pixfmt: p010le (Vulkan ID: 1000156013) F:/dev/ffmpeg/libavutil/vulkan.c:406:55: runtime error: applying zero offset to null pointer #0 0x7ff8173295b2 in ff_vk_exec_pool_init F:/dev/ffmpeg/libavutil/vulkan.c:406:55 #1 0x7ff816b4c702 in vulkan_frames_init F:/dev/ffmpeg/libavutil/hwcontext_vulkan.c:2326:11 #2 0x7ff816af16a2 in av_hwframe_ctx_init F:/dev/ffmpeg/libavutil/hwcontext.c:364:15 #3 0x7ff651974d8f in init_generic_hwaccel F:/dev/mpv/build/../video/decode/vd_lavc.c:889:13 #4 0x7ff651974d8f in get_format_hwdec F:/dev/mpv/build/../video/decode/vd_lavc.c:934:17 #5 0x7fffe26561ad in ff_get_format F:/dev/ffmpeg/libavcodec/decode.c:1198:23 #6 0x7fffe44f4b9d in get_format F:/dev/ffmpeg/libavcodec/hevcdec.c:524:12 #7 0x7fffe44d0f56 in hls_slice_header F:/dev/ffmpeg/libavcodec/hevcdec.c:637:19 #8 0x7fffe44c9480 in decode_nal_unit F:/dev/ffmpeg/libavcodec/hevcdec.c:3079:15 #9 0x7fffe44c3600 in decode_nal_units F:/dev/ffmpeg/libavcodec/hevcdec.c:3268:15 #10 0x7fffe44adf60 in hevc_decode_frame F:/dev/ffmpeg/libavcodec/hevcdec.c:3410:14 #11 0x7fffe2669344 in decode_simple_internal F:/dev/ffmpeg/libavcodec/decode.c:287:15 #12 0x7fffe266794b in decode_simple_receive_frame F:/dev/ffmpeg/libavcodec/decode.c:533:15 #13 0x7fffe2646609 in decode_receive_frame_internal F:/dev/ffmpeg/libavcodec/decode.c:553:15 #14 0x7fffe2645f9b in avcodec_send_packet F:/dev/ffmpeg/libavcodec/decode.c:641:15 #15 0x7ff65196b643 in send_packet F:/dev/mpv/build/../video/decode/vd_lavc.c:1095:15 #16 0x7ff65168405d in lavc_process F:/dev/mpv/build/../filters/f_decoder_wrapper.c:1299:24 #17 0x7ff6516d21a9 in mp_filter_graph_run F:/dev/mpv/build/../filters/filter.c:262:13 #18 0x7ff651897316 in run_playloop F:/dev/mpv/build/../player/playloop.c:1245:9 #19 0x7ff6518694ef in play_current_file F:/dev/mpv/build/../player/loadfile.c:1779:9 #20 0x7ff6518629c9 in mp_play_files F:/dev/mpv/build/../player/loadfile.c:1965:13 #21 0x7ff65187a88c in mpv_main F:/dev/mpv/build/../player/main.c:438:9 #22 0x7ff651af9b19 in main F:/dev/mpv/build/../osdep/main-fn-win.c:74:15 #23 0x7ff651491314 in __tmainCRTStartup C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:267:15 #24 0x7ff651491365 in .l_start C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:188:9 #25 0x7ff871e07613 (C:\Windows\System32\KERNEL32.DLL+0x180017613) #26 0x7ff8723e26b0 (C:\Windows\SYSTEM32\ntdll.dll+0x1800526b0) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior F:/dev/ffmpeg/libavutil/vulkan.c:406:55 in F:/dev/ffmpeg/libavutil/vulkan.c:406:55: runtime error: applying zero offset to null pointer #0 0x7fffe88e90e2 in ff_vk_exec_pool_init F:/dev/ffmpeg/libavutil/vulkan.c:406:55 #1 0x7fffe8955704 in ff_vk_decode_init F:/dev/ffmpeg/libavcodec/vulkan_decode.c:1131:11 #2 0x7fffe265912e in hwaccel_init F:/dev/ffmpeg/libavcodec/decode.c:1143:15 #3 0x7fffe2657dbb in ff_get_format F:/dev/ffmpeg/libavcodec/decode.c:1281:19 #4 0x7fffe44f4b9d in get_format F:/dev/ffmpeg/libavcodec/hevcdec.c:524:12 #5 0x7fffe44d0f56 in hls_slice_header F:/dev/ffmpeg/libavcodec/hevcdec.c:637:19 #6 0x7fffe44c9480 in decode_nal_unit F:/dev/ffmpeg/libavcodec/hevcdec.c:3079:15 #7 0x7fffe44c3600 in decode_nal_units F:/dev/ffmpeg/libavcodec/hevcdec.c:3268:15 #8 0x7fffe44adf60 in hevc_decode_frame F:/dev/ffmpeg/libavcodec/hevcdec.c:3410:14 #9 0x7fffe2669344 in decode_simple_internal F:/dev/ffmpeg/libavcodec/decode.c:287:15 #10 0x7fffe266794b in decode_simple_receive_frame F:/dev/ffmpeg/libavcodec/decode.c:533:15 #11 0x7fffe2646609 in decode_receive_frame_internal F:/dev/ffmpeg/libavcodec/decode.c:553:15 #12 0x7fffe2645f9b in avcodec_send_packet F:/dev/ffmpeg/libavcodec/decode.c:641:15 #13 0x7ff65196b643 in send_packet F:/dev/mpv/build/../video/decode/vd_lavc.c:1095:15 #14 0x7ff65168405d in lavc_process F:/dev/mpv/build/../filters/f_decoder_wrapper.c:1299:24 #15 0x7ff6516d21a9 in mp_filter_graph_run F:/dev/mpv/build/../filters/filter.c:262:13 #16 0x7ff651897316 in run_playloop F:/dev/mpv/build/../player/playloop.c:1245:9 #17 0x7ff6518694ef in play_current_file F:/dev/mpv/build/../player/loadfile.c:1779:9 #18 0x7ff6518629c9 in mp_play_files F:/dev/mpv/build/../player/loadfile.c:1965:13 #19 0x7ff65187a88c in mpv_main F:/dev/mpv/build/../player/main.c:438:9 #20 0x7ff651af9b19 in main F:/dev/mpv/build/../osdep/main-fn-win.c:74:15 #21 0x7ff651491314 in __tmainCRTStartup C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:267:15 #22 0x7ff651491365 in .l_start C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:188:9 #23 0x7ff871e07613 (C:\Windows\System32\KERNEL32.DLL+0x180017613) #24 0x7ff8723e26b0 (C:\Windows\SYSTEM32\ntdll.dll+0x1800526b0) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior F:/dev/ffmpeg/libavutil/vulkan.c:406:55 in [ffmpeg/video] hevc: Allocating 40161280 bytes in bind index 0 for video session [ffmpeg/video] hevc: Vulkan decoder initialization sucessful [ffmpeg/video] hevc: Decoding frame, 1574 bytes, 1 slices [ffmpeg/video] hevc: Decoding frame, 119 bytes, 1 slices [ffmpeg/video] hevc: Decoding frame, 118 bytes, 1 slices [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST [ffmpeg/video] hevc: hardware accelerator failed to decode picture Error while decoding frame (hardware decoding)! [ffmpeg/video] hevc: Decoding frame, 120 bytes, 1 slices [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST [ffmpeg/video] hevc: hardware accelerator failed to decode picture Error while decoding frame (hardware decoding)! [ffmpeg/video] hevc: Could not find ref with POC 4 [ffmpeg] av_log callback called with bad parameters (NULL AVClass). [ffmpeg] This is a bug in one of Libav/FFmpeg libraries used. [ffmpeg] Unable to submit command buffer: VK_ERROR_DEVICE_LOST [ffmpeg/video] hevc: get_buffer() failed [ffmpeg/video] hevc: Error constructing the frame RPS. [ffmpeg/video] hevc: Error parsing NAL unit #2. Error while decoding frame (hardware decoding)! ```
vulkan 1080p h.264 VVL (different msg, because I had 0x30f4ac70 not excluded in libplacebo before) ``` [ffmpeg] AVHWDeviceContext: Using device: AMD Radeon RX 5700 XT [ffmpeg] AVHWDeviceContext: Alignments: [ffmpeg] AVHWDeviceContext: optimalBufferCopyRowPitchAlignment: 1 [ffmpeg] AVHWDeviceContext: minMemoryMapAlignment: 64 [ffmpeg] AVHWDeviceContext: nonCoherentAtomSize: 128 [ffmpeg] AVHWDeviceContext: minImportedHostPointerAlignment: 4096 [ffmpeg] AVHWDeviceContext: Using queue family 0 (queues: 1) for graphics encode [ffmpeg] AVHWDeviceContext: Using queue family 2 (queues: 1) for transfers [ffmpeg] AVHWDeviceContext: Using queue family 1 (queues: 1) for compute [ffmpeg] AVHWDeviceContext: Using queue family 4 (queues: 1) for decode [ffmpeg/video] h264: Decoder capabilities for h264 profile "High": [ffmpeg/video] h264: Maximum level: 0 (stream 40) [ffmpeg/video] h264: Width: from 16 to 4096 [ffmpeg/video] h264: Height: from 16 to 4096 [ffmpeg/video] h264: Width alignment: 16 [ffmpeg/video] h264: Height alignment: 16 [ffmpeg/video] h264: Bitstream offset alignment: 4096 [ffmpeg/video] h264: Bitstream size alignment: 4096 [ffmpeg/video] h264: Maximum references: 17 [ffmpeg/video] h264: Maximum active references: 17 [ffmpeg/video] h264: Codec header version: 0.0.0 (driver), 1.0.0 (compiled) [ffmpeg/video] h264: Decode modes: dedicated_dpb [ffmpeg/video] h264: Capability flags: none [ffmpeg/video] h264: Chosen frame pixfmt: nv12 (Vulkan ID: 1000156003) F:/dev/ffmpeg/libavutil/vulkan.c:406:55: runtime error: applying zero offset to null pointer #0 0x7ff8173295b2 in ff_vk_exec_pool_init F:/dev/ffmpeg/libavutil/vulkan.c:406:55 #1 0x7ff816b4c702 in vulkan_frames_init F:/dev/ffmpeg/libavutil/hwcontext_vulkan.c:2326:11 #2 0x7ff816af16a2 in av_hwframe_ctx_init F:/dev/ffmpeg/libavutil/hwcontext.c:364:15 #3 0x7ff651974d8f in init_generic_hwaccel F:/dev/mpv/build/../video/decode/vd_lavc.c:889:13 #4 0x7ff651974d8f in get_format_hwdec F:/dev/mpv/build/../video/decode/vd_lavc.c:934:17 #5 0x7fffe26561ad in ff_get_format F:/dev/ffmpeg/libavcodec/decode.c:1198:23 #6 0x7fffe3b1934a in get_pixel_format F:/dev/ffmpeg/libavcodec/h264_slice.c:908:12 #7 0x7fffe3affdd5 in h264_init_ps F:/dev/ffmpeg/libavcodec/h264_slice.c:1147:20 #8 0x7fffe3ac2d30 in h264_field_start F:/dev/ffmpeg/libavcodec/h264_slice.c:1398:11 #9 0x7fffe3ab5480 in ff_h264_queue_decode_slice F:/dev/ffmpeg/libavcodec/h264_slice.c:2135:15 #10 0x7fffe3bb5ecd in decode_nal_units F:/dev/ffmpeg/libavcodec/h264dec.c:641:24 #11 0x7fffe3bace45 in h264_decode_frame F:/dev/ffmpeg/libavcodec/h264dec.c:1015:17 #12 0x7fffe2669344 in decode_simple_internal F:/dev/ffmpeg/libavcodec/decode.c:287:15 #13 0x7fffe266794b in decode_simple_receive_frame F:/dev/ffmpeg/libavcodec/decode.c:533:15 #14 0x7fffe2646609 in decode_receive_frame_internal F:/dev/ffmpeg/libavcodec/decode.c:553:15 #15 0x7fffe2645f9b in avcodec_send_packet F:/dev/ffmpeg/libavcodec/decode.c:641:15 #16 0x7ff65197161b in init_avctx F:/dev/mpv/build/../video/decode/vd_lavc.c:777:9 #17 0x7ff6519690ca in reinit F:/dev/mpv/build/../video/decode/vd_lavc.c:612:5 #18 0x7ff651962cc4 in create.llvm.14630122091301803147 F:/dev/mpv/build/../video/decode/vd_lavc.c:1362:5 #19 0x7ff65167d2c0 in reinit_decoder F:/dev/mpv/build/../filters/f_decoder_wrapper.c:450:22 #20 0x7ff65167d2c0 in mp_decoder_wrapper_reinit F:/dev/mpv/build/../filters/f_decoder_wrapper.c:478:16 #21 0x7ff6518af19d in init_video_decoder F:/dev/mpv/build/../player/video.c:188:10 #22 0x7ff6518afd87 in reinit_video_chain_src F:/dev/mpv/build/../player/video.c:257:14 #23 0x7ff651868b35 in play_current_file F:/dev/mpv/build/../player/loadfile.c:1706:5 #24 0x7ff6518629c9 in mp_play_files F:/dev/mpv/build/../player/loadfile.c:1965:13 #25 0x7ff65187a88c in mpv_main F:/dev/mpv/build/../player/main.c:438:9 #26 0x7ff651af9b19 in main F:/dev/mpv/build/../osdep/main-fn-win.c:74:15 #27 0x7ff651491314 in __tmainCRTStartup C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:267:15 #28 0x7ff651491365 in .l_start C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:188:9 #29 0x7ff871e07613 (C:\Windows\System32\KERNEL32.DLL+0x180017613) #30 0x7ff8723e26b0 (C:\Windows\SYSTEM32\ntdll.dll+0x1800526b0) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior F:/dev/ffmpeg/libavutil/vulkan.c:406:55 in F:/dev/ffmpeg/libavutil/vulkan.c:406:55: runtime error: applying zero offset to null pointer #0 0x7fffe88e90e2 in ff_vk_exec_pool_init F:/dev/ffmpeg/libavutil/vulkan.c:406:55 #1 0x7fffe8955704 in ff_vk_decode_init F:/dev/ffmpeg/libavcodec/vulkan_decode.c:1131:11 #2 0x7fffe265912e in hwaccel_init F:/dev/ffmpeg/libavcodec/decode.c:1143:15 #3 0x7fffe2657dbb in ff_get_format F:/dev/ffmpeg/libavcodec/decode.c:1281:19 #4 0x7fffe3b1934a in get_pixel_format F:/dev/ffmpeg/libavcodec/h264_slice.c:908:12 #5 0x7fffe3affdd5 in h264_init_ps F:/dev/ffmpeg/libavcodec/h264_slice.c:1147:20 #6 0x7fffe3ac2d30 in h264_field_start F:/dev/ffmpeg/libavcodec/h264_slice.c:1398:11 #7 0x7fffe3ab5480 in ff_h264_queue_decode_slice F:/dev/ffmpeg/libavcodec/h264_slice.c:2135:15 #8 0x7fffe3bb5ecd in decode_nal_units F:/dev/ffmpeg/libavcodec/h264dec.c:641:24 #9 0x7fffe3bace45 in h264_decode_frame F:/dev/ffmpeg/libavcodec/h264dec.c:1015:17 #10 0x7fffe2669344 in decode_simple_internal F:/dev/ffmpeg/libavcodec/decode.c:287:15 #11 0x7fffe266794b in decode_simple_receive_frame F:/dev/ffmpeg/libavcodec/decode.c:533:15 #12 0x7fffe2646609 in decode_receive_frame_internal F:/dev/ffmpeg/libavcodec/decode.c:553:15 #13 0x7fffe2645f9b in avcodec_send_packet F:/dev/ffmpeg/libavcodec/decode.c:641:15 #14 0x7ff65197161b in init_avctx F:/dev/mpv/build/../video/decode/vd_lavc.c:777:9 #15 0x7ff6519690ca in reinit F:/dev/mpv/build/../video/decode/vd_lavc.c:612:5 #16 0x7ff651962cc4 in create.llvm.14630122091301803147 F:/dev/mpv/build/../video/decode/vd_lavc.c:1362:5 #17 0x7ff65167d2c0 in reinit_decoder F:/dev/mpv/build/../filters/f_decoder_wrapper.c:450:22 #18 0x7ff65167d2c0 in mp_decoder_wrapper_reinit F:/dev/mpv/build/../filters/f_decoder_wrapper.c:478:16 #19 0x7ff6518af19d in init_video_decoder F:/dev/mpv/build/../player/video.c:188:10 #20 0x7ff6518afd87 in reinit_video_chain_src F:/dev/mpv/build/../player/video.c:257:14 #21 0x7ff651868b35 in play_current_file F:/dev/mpv/build/../player/loadfile.c:1706:5 #22 0x7ff6518629c9 in mp_play_files F:/dev/mpv/build/../player/loadfile.c:1965:13 #23 0x7ff65187a88c in mpv_main F:/dev/mpv/build/../player/main.c:438:9 #24 0x7ff651af9b19 in main F:/dev/mpv/build/../osdep/main-fn-win.c:74:15 #25 0x7ff651491314 in __tmainCRTStartup C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:267:15 #26 0x7ff651491365 in .l_start C:/M/B/src/mingw-w64/mingw-w64-crt/crt/crtexe.c:188:9 #27 0x7ff871e07613 (C:\Windows\System32\KERNEL32.DLL+0x180017613) #28 0x7ff8723e26b0 (C:\Windows\SYSTEM32\ntdll.dll+0x1800526b0) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior F:/dev/ffmpeg/libavutil/vulkan.c:406:55 in [vo/gpu-next/libplacebo] vk Validation Error: [ VUID-VkVideoSessionCreateInfoKHR-pStdHeaderVersion-07190 ] Object 0: handle = 0x1320d5fa68a0, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x53b6151c | vkCreateVideoSessionKHR(): unsupported Video Std header name 'VK_STD_vulkan_video_codec_h264_decode' specified in pCreateInfo->pStdHeaderVersion->extensionName, expected '' The Vulkan spec states: pStdHeaderVersion->extensionName must match VkVideoCapabilitiesKHR::stdHeaderVersion.extensionName, as returned by vkGetPhysicalDeviceVideoCapabilitiesKHR for the video profile specified by pVideoProfile (https://vulkan.lunarg.com/doc/view/1.3.250.0/windows/1.3-extensions/vkspec.html#VUID-VkVideoSessionCreateInfoKHR-pStdHeaderVersion-07190) [vo/gpu-next/libplacebo] using VkDevice: anon (0x1320d5fa68a0) [vo/gpu-next/libplacebo] Backtrace: [vo/gpu-next/libplacebo] #0 0x7ff812fe42a8 in vk_dbg_utils_cb+0xb28 F:\dev\libplacebo\src\vulkan\context.c:535+0xa [vo/gpu-next/libplacebo] #1 0x7ff810b97b1c in debug_log_msg+0xb8c C:\j\msdk\build\Vulkan-ValidationLayers\repo\layers\error_message\logging.cpp:198+0x12 [vo/gpu-next/libplacebo] #2 0x7ff810b99313 in LogMsg+0xc53 C:\j\msdk\build\Vulkan-ValidationLayers\repo\layers\error_message\logging.cpp:437+0x30 [vo/gpu-next/libplacebo] #3 0x7ff8106c8d76 in ValidationObject::LogError+0x36 C:\j\msdk\build\Vulkan-ValidationLayers\repo\layers\vulkan\generated\chassis.h:4073+0x2d [vo/gpu-next/libplacebo] #4 0x7ff8107fdd01 in CoreChecks::PreCallValidateCreateVideoSessionKHR+0x8e1 C:\j\msdk\build\Vulkan-ValidationLayers\repo\layers\core_checks\cc_video.cpp:866+0xe6 [vo/gpu-next/libplacebo] #5 0x7ff81088d50b in vulkan_layer_chassis::CreateVideoSessionKHR+0xcb C:\j\msdk\build\Vulkan-ValidationLayers\repo\layers\vulkan\generated\chassis.cpp:6312+0x19 [vo/gpu-next/libplacebo] #6 0x7fffe89a6fdf in av_vorbis_parse_init+0x12d546f (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x7746fdf) (0x187746fdf) [vo/gpu-next/libplacebo] #7 0x7fffe89557ad in av_vorbis_parse_init+0x1283c3d (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x76f57ad) (0x1876f57ad) [vo/gpu-next/libplacebo] #8 0x7fffe265912e in avcodec_get_hw_frames_parameters+0x57ee (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x13f912e) (0x1813f912e) [vo/gpu-next/libplacebo] #9 0x7fffe2657dbb in avcodec_get_hw_frames_parameters+0x447b (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x13f7dbb) (0x1813f7dbb) [vo/gpu-next/libplacebo] #10 0x7fffe3b1934a in avcodec_default_get_buffer2+0x8e216a (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x28b934a) (0x1828b934a) [vo/gpu-next/libplacebo] #11 0x7fffe3affdd5 in avcodec_default_get_buffer2+0x8c8bf5 (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x289fdd5) (0x18289fdd5) [vo/gpu-next/libplacebo] #12 0x7fffe3ac2d30 in avcodec_default_get_buffer2+0x88bb50 (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x2862d30) (0x182862d30) [vo/gpu-next/libplacebo] #13 0x7fffe3ab5480 in avcodec_default_get_buffer2+0x87e2a0 (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x2855480) (0x182855480) [vo/gpu-next/libplacebo] #14 0x7fffe3bb5ecd in avpriv_h264_has_num_reorder_frames+0x1f52d (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x2955ecd) (0x182955ecd) [vo/gpu-next/libplacebo] #15 0x7fffe3bace45 in avpriv_h264_has_num_reorder_frames+0x164a5 (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x294ce45) (0x18294ce45) [vo/gpu-next/libplacebo] #16 0x7fffe2669344 in avcodec_get_hw_frames_parameters+0x15a04 (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x1409344) (0x181409344) [vo/gpu-next/libplacebo] #17 0x7fffe266794b in avcodec_get_hw_frames_parameters+0x1400b (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x140794b) (0x18140794b) [vo/gpu-next/libplacebo] #18 0x7fffe2646609 in avcodec_send_packet+0x1519 (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x13e6609) (0x1813e6609) [vo/gpu-next/libplacebo] #19 0x7fffe2645f9b in avcodec_send_packet+0xeab (C:\MSYS\home\Kacper\install\bin\avcodec-60.dll+0x13e5f9b) (0x1813e5f9b) [vo/gpu-next/libplacebo] #20 0x7ff65197161b in mpv_stream_cb_add_ro+0x19a6eb (C:\Apps\mpv\mpv.exe+0x4e161b) (0x1404e161b) [vo/gpu-next/libplacebo] #21 0x7ff6519690ca in mpv_stream_cb_add_ro+0x19219a (C:\Apps\mpv\mpv.exe+0x4d90ca) (0x1404d90ca) [vo/gpu-next/libplacebo] #22 0x7ff651962cc4 in mpv_stream_cb_add_ro+0x18bd94 (C:\Apps\mpv\mpv.exe+0x4d2cc4) (0x1404d2cc4) [vo/gpu-next/libplacebo] #23 0x7ff65167d2c0 (C:\Apps\mpv\mpv.exe+0x1ed2c0) (0x1401ed2c0) [vo/gpu-next/libplacebo] #24 0x7ff6518af19d in mpv_stream_cb_add_ro+0xd826d (C:\Apps\mpv\mpv.exe+0x41f19d) (0x14041f19d) [vo/gpu-next/libplacebo] #25 0x7ff6518afd87 in mpv_stream_cb_add_ro+0xd8e57 (C:\Apps\mpv\mpv.exe+0x41fd87) (0x14041fd87) [vo/gpu-next/libplacebo] #26 0x7ff651868b35 in mpv_stream_cb_add_ro+0x91c05 (C:\Apps\mpv\mpv.exe+0x3d8b35) (0x1403d8b35) [vo/gpu-next/libplacebo] #27 0x7ff6518629c9 in mpv_stream_cb_add_ro+0x8ba99 (C:\Apps\mpv\mpv.exe+0x3d29c9) (0x1403d29c9) [vo/gpu-next/libplacebo] #28 0x7ff65187a88c in mpv_stream_cb_add_ro+0xa395c (C:\Apps\mpv\mpv.exe+0x3ea88c) (0x1403ea88c) [vo/gpu-next/libplacebo] #29 0x7ff651af9b19 in mpv_render_context_get_info+0x5b439 (C:\Apps\mpv\mpv.exe+0x669b19) (0x140669b19) [vo/gpu-next/libplacebo] #30 0x7ff651491314 (C:\Apps\mpv\mpv.exe+0x1314) (0x140001314) [vo/gpu-next/libplacebo] #31 0x7ff651491365 (C:\Apps\mpv\mpv.exe+0x1365) (0x140001365) [vo/gpu-next/libplacebo] #32 0x7ff871e07613 in BaseThreadInitThunk+0x13 (C:\Windows\System32\KERNEL32.DLL+0x17613) (0x180017613) [vo/gpu-next/libplacebo] #33 0x7ff8723e26b0 in RtlUserThreadStart+0x20 (C:\Windows\SYSTEM32\ntdll.dll+0x526b0) (0x1800526b0) ```
cyanreg commented 1 year ago

Thanks, I don't see anything bad. Guess they didn't test their driver well at all on anything but the utterly useless "example" program. Do report it to them if you can.

kasper93 commented 1 year ago

I also tested on RDNA2 integrated GPU, same result.

Jules-A commented 1 year ago

23.7.1 has released with Vulkan HW Decode but still no support on Polaris :(

Artoriuz commented 1 year ago

23.7.1 has released with Vulkan HW Decode but still no support on Polaris :(

It doesn't work on supported hardware either =)

cyanreg commented 1 year ago

I've told AMD it's broken, hopefully they'll fix it.

alakittty commented 1 year ago

23.7.1 has released with Vulkan HW Decode but still no support on Polaris :(

It doesn't work on supported hardware either =)

can confirm doesn't work on my supported igpu either

infinity0 commented 1 year ago

Got this working on Debian on Radeon RX 580 (POLARIS10) with Mesa RADV driver with the commits stated in the OP, except that I added MR 22202 and MR 23304 on top of Debian's Mesa 23.1.3 rather than using upstream mesa master branch directly.

Anyway, it works with 1080p HD video but not 8K video:

$ ffprobe PXL_[..].mp4
[..]
  Stream #0:1[0x2](eng): Video: hevc (Main) (hev1 / 0x31766568), yuvj420p(pc, bt470bg/bt470bg/smpte170m), 1920x1080, 12041 kb/s, SAR 1:1 DAR 16:9, 30.18 fps, 60 tbr, 90k tbn (default)
[..]
$ ./build/mpv [..]
[..]
[vd] Trying hardware decoding via hevc-vulkan.
[vd] Selected codec: hevc (HEVC (High Efficiency Video Coding))
[..]
[vd] Pixel formats supported by decoder: vaapi vdpau cuda vulkan yuvj420p
[vd] Codec profile: Main (0x1)
[ffmpeg/video] hevc: Decoder capabilities for hevc profile "Main":
[ffmpeg/video] hevc:     Maximum level: 51 (stream 150)
[ffmpeg/video] hevc:     Width: from 16 to 4096
[ffmpeg/video] hevc:     Height: from 16 to 4096
[ffmpeg/video] hevc:     Width alignment: 16
[ffmpeg/video] hevc:     Height alignment: 16
[ffmpeg/video] hevc:     Bitstream offset alignment: 128
[ffmpeg/video] hevc:     Bitstream size alignment: 128
[ffmpeg/video] hevc:     Maximum references: 17
[ffmpeg/video] hevc:     Maximum active references: 8
[ffmpeg/video] hevc:     Codec header version: 1.0.0 (driver), 1.0.0 (compiled)
[ffmpeg/video] hevc:     Decode modes: dedicated_dpb
[ffmpeg/video] hevc:     Capability flags: none
[ffmpeg/video] hevc: Chosen frame pixfmt: nv12 (Vulkan ID: 1000156003)
[vd] Requesting pixfmt 'vulkan' from decoder.
[ffmpeg/video] hevc: Allocating 131072 bytes in bind index 0 for video session
[ffmpeg/video] hevc: Allocating 21204992 bytes in bind index 1 for video session
[ffmpeg/video] hevc: Vulkan decoder initialization sucessful
[ffmpeg/video] hevc: Decoding frame, 203715 bytes, 3 slices
Using hardware decoding (vulkan).
[vd] Decoder format: 1920x1080 [0:1] vulkan[nv12] bt.601/bt.601-625/bt.1886/full/auto CL=mpeg2/4/h264
[vd] Using container aspect ratio.
[..]
$ ffprobe GX01_[..].mp4
[..]
  Stream #0:0[0x1](eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 5312x2988 [SAR 1:1 DAR 16:9], 59867 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
[..]
$ ./build/mpv [..]
[..]
[vd] Trying hardware decoding via hevc-vulkan.
[vd] Selected codec: hevc (HEVC (High Efficiency Video Coding))
[..]
[vd] Pixel formats supported by decoder: vaapi vdpau cuda vulkan yuvj420p
[vd] Codec profile: Main (0x1)
[ffmpeg/video] hevc: Decoder capabilities for hevc profile "Main":
[ffmpeg/video] hevc:     Maximum level: 51 (stream 180)
[ffmpeg/video] hevc:     Width: from 16 to 4096
[ffmpeg/video] hevc:     Height: from 16 to 4096
[ffmpeg/video] hevc:     Width alignment: 16
[ffmpeg/video] hevc:     Height alignment: 16
[ffmpeg/video] hevc:     Bitstream offset alignment: 128
[ffmpeg/video] hevc:     Bitstream size alignment: 128
[ffmpeg/video] hevc:     Maximum references: 17
[ffmpeg/video] hevc:     Maximum active references: 8
[ffmpeg/video] hevc:     Codec header version: 1.0.0 (driver), 1.0.0 (compiled)
[ffmpeg/video] hevc:     Decode modes: dedicated_dpb
[ffmpeg/video] hevc:     Capability flags: none
[vd] Hardware decoding of this stream is unsupported?
[vd] Requesting pixfmt 'yuvj420p' from decoder.
[af] [in] 48000Hz stereo 2ch floatp
[af] [userspeed] 48000Hz stereo 2ch floatp
[af] [userspeed] (disabled)
[af] [convert] 48000Hz stereo 2ch floatp
[vd] Falling back to software decoding.
[..]
[.. CPU fan goes brrrrrrrr... shit slows down ... desync ..]
AV: 00:00:28 / 00:01:57 (24%) A-V:  0.473 Dropped: 134

Audio/Video desynchronisation detected! Possible reasons include too slow
hardware, temporary CPU spikes, broken drivers, and broken files. Audio
position will not match to the video (see A-V status field).
[..]

Actually even with --hwdec=vaapi I got:

[ffmpeg/video] hevc: Hardware does not support image size 5312x2992 (constraints: width 0-4096 height 0-4096).

Is there any way to make my 8K video play on my GPU, can it be fixed in software/firmware, are we waiting for future code to make it work, or is it just physically not supported by my hardware Radeon RX 580 (POLARIS10)?

philipl commented 1 year ago

Is there any way to make my 8K video play on my GPU, can it be fixed in software/firmware, are we waiting for future code to make it work, or is it just physically not supported by my hardware Radeon RX 580 (POLARIS10)?

No. Your hardware is limited to decoding up to 4K, which is to be expected from that hardware generation.

infinity0 commented 1 year ago

Damn that's a shame. Now to think about whether I want to fork out another $400 on a graphics card. :(

My video is actually 5.3k x 3k px. Not even a theoretical hacky way to make it "fit" into the 4k x 4k space limited by the existing old card?

philipl commented 1 year ago

Damn that's a shame. Now to think about whether I want to fork out another $400 on a graphics card. :(

My video is actually 5.3k x 3k px. Not even a theoretical hacky way to make it "fit" into the 4k x 4k space limited by the existing old card?

No. You can't do that - the layout is part of the encoded structure. As for hardware, you can get a significant upgrade moving to a 6700xt or 6750xt for significantly less than $400.

Infinitytreacher commented 1 year ago

What about VP9 ?

https://www.techgoing.com/vulkan-video-to-add-support-for-vp9-and-av1-format-decoding-encoding-in-2023/

Infinitytreacher commented 1 year ago

@infinity0 Interesting. But that's old article. What's the development status of it now, if you know ?

that's not me, you just pinged a random guy Outta the blue

akiirui commented 1 year ago

23.7.1 has released with Vulkan HW Decode but still no support on Polaris :(

I tried to use it, but my computer froze for a few minutes when I started playing a video, then restarted. (7900XTX)

cyanreg commented 1 year ago

Did it actually decode something?

ghost commented 1 year ago

I seriously doubt that Intel and AMD released the Vulkan Video driver without testing it at all and everything doesn't work.

llyyr commented 1 year ago

Prepare to be very surprised then, it also doesn't work on my 6600 XT on Windows fwiw

Infinitytreacher commented 1 year ago

I seriously doubt that Intel and AMD released the Vulkan Video driver without testing it at all and everything doesn't work.

Apparently they betrayed your expectations?

llyyr commented 1 year ago

a new libplacebo is already released and ffmpeg will cut a new release soonish, may as well update all of them at once. Mesa should also contain all the necessary parts as of 23.2

laichiaheng commented 1 year ago

I don't see --hwdec=vulkan in the latest release, but it can decode with VAAPI.

netExtra commented 1 year ago

  • Nvidia

    • Both H.264 and H.265 are supported

    • Vulkan filters are supported, but not for streams that are decoded with Vulkan video decoding (but of course).

    • Filters willl work with content that is uploaded to Vulkan after being decoded by nvdec or software decoding.

    • eg: mpv --vo=gpu-next --gpu-api=vulkan --hwdec=nvdec --vf=format=vulkan

'hwdec=vulkan' worked for me using Nvidia driver 536.67 on Windows running mpv 0.36 built with ffmpeg version git-2023-07-30.

philipl commented 1 year ago

'hwdec=vulkan' worked for me using Nvidia driver 536.67 on Windows running mpv 0.36 built with ffmpeg version git-2023-07-30.

Glad to hear. We'll have to see when we get a Linux driver release with the fix. I haven't been able to try 535.86.05 yet.

netExtra commented 1 year ago

The vulkan hwdec on nvidia has always worked, the problem is that the vk hwdec+filter never works. Not sure if the fix is included in R545, my GPU PCI ID is not in there to test it

The 'hwdec=vulkan' command did not work before. Vulkan as detailed in this user guide only worked by using the 'hwdec=nvdec' workaround. As far as I can tell, everything else works. Is there anything in particular to check?

netExtra commented 1 year ago

vulkan hwdec is always available since 535.98, but the vk filters don't work. set vf hflip_vulkan, if everything works, you should see the video has been flipped

I knew my driver (earlier than .98) was an issue after I upgraded it which is why I specified it in my post. Anyway "hflip" works under hwdec_vulkan. "hflip_vulkan" does not.

hooke007 commented 1 year ago

Vulkan filters are supported, but not for streams that are decoded with Vulkan video decoding (but of course).

hflip is irrelevant. Because what you quoted is tallking about vulkan-filter. hflip_vulkan is a vulkan-filter while hflip is not.

netExtra commented 1 year ago

Vulkan filters are supported, but not for streams that are decoded with Vulkan video decoding (but of course).

hflip is irrelevant. Because what you quoted is tallking about vulkan-filter. hflip_vulkan is a vulkan-filter while hflip is not.

Error message: [ffmpeg] Impossible to convert between the formats supported by the filter 'mpv_scr_default_in' and the filter 'auto_scale_0'

charlie-ht commented 1 year ago

I seriously doubt that Intel and AMD released the Vulkan Video driver without testing it at all and everything doesn't work.

Apparently they betrayed your expectations?

Sure, at first, I thought Vulkan Video was the future, but look at it now, Google explicitly banned Vulkan Video in the Android CDD, Chrome Media said they weren't interested in it, there are countless problems with all the Vulkan Video drivers on Windows, Khronos is still delaying AV1 and VP9, and mesa is the only one working properly.

Please can you reference the explicit "ban"?

Also for Chrome Media? I understand there could be nice wins for VK Video on mobile when implementations start to appear there. Chromium at least seem they would be open to welcoming patches: https://bugs.chromium.org/p/chromium/issues/detail?id=1200559

Khronos isn't delaying AV1 or VP9, AV1 is in progress (there exists a Mesa extension already), and VP9 is on the future roadmap.

llyyr commented 1 year ago

Please can you reference the explicit "ban"?

https://source.android.com/docs/compatibility/13/android-13-cdd#7142_vulkan

[C-1-11] MUST NOT enumerate support for the VK_KHR_video_queue, VK_KHR_video_decode_queue, or VK_KHR_video_encode_queue extensions.

JoonasPitk commented 1 year ago

Vulkan filters are supported, but not for streams that are decoded with Vulkan video decoding (but of course).

hflip is irrelevant. Because what you quoted is tallking about vulkan-filter. hflip_vulkan is a vulkan-filter while hflip is not.

hflip_vulkan doesn't seem to exist for me. :thinking: I do meet requirements set by mpv for Vulkan Video decoding for Nvidia on Linux. Or is the hwdec=nvdec still in the realm of yank, while hwdec=vulkan is supported in 0.36.0?

cyanreg commented 1 year ago

https://github.com/FFmpeg/FFmpeg/commit/c0709706ddda0d1f079d4020047bcdeb0bc4a7f1 may have fixed some H264 streams on Nvidia

Obegg commented 1 year ago

@arm64-v9a I have the same issue and that's reason number 2 why I don't use hwdec=vulkan (I didn't get to test the new patch, will test it tomorrow if there's a build available to download) Is there an open issue regarding this specific case? I would like to follow up on this.

The other issue I don't use vulkan is the stuttering issue I've mentioned to you before. (This reminds me - try to reproduce that stuttering issue with OSD turned on, I did not see you mention anything about OSD)

charlie-ht commented 1 year ago

FFmpeg/FFmpeg@c070970 may have fixed some H264 streams on Nvidia

This sample still has problems

!!!! This sample is 2s cut from the beginning of an adult movie and is not NSFW, I couldn't find any other samples with this problem, but it's still recommended to check it without other people around !!!!

What exactly is the problem you are seeing? The sample decodes fine on my GTX 3050, both with FFmpeg and the NVIDIA Vulkan sample decoder.

FFmpeg here is at 2cd09fec9f8d3b16d03e762dec6001cc0b792659.

Command line to verify,

ffmpeg_g -hide_banner -loglevel verbose -y -init_hw_device vulkan=vk:0,debug=0 -hwaccel_output_format vulkan -hwaccel vulkan -i /home/cturner/fail.webm -fps_mode passthrough -vf hwdownload,format=nv12,format=pix_fmts=yuv420p /tmp/ffmpeg.yuv
charlie-ht commented 1 year ago

FFmpeg/FFmpeg@c070970 may have fixed some H264 streams on Nvidia

This sample still has problems !!!! This sample is 2s cut from the beginning of an adult movie and is not NSFW, I couldn't find any other samples with this problem, but it's still recommended to check it without other people around !!!!

What exactly is the problem you are seeing? The sample decodes fine on my GTX 3050, both with FFmpeg and the NVIDIA Vulkan sample decoder. FFmpeg here is at 2cd09fec9f8d3b16d03e762dec6001cc0b792659. Command line to verify,

ffmpeg_g -hide_banner -loglevel verbose -y -init_hw_device vulkan=vk:0,debug=0 -hwaccel_output_format vulkan -hwaccel vulkan -i /home/cturner/fail.webm -fps_mode passthrough -vf hwdownload,format=nv12,format=pix_fmts=yuv420p /tmp/ffmpeg.yuv

You can of course decode this video without errors, but you will notice that the output is flawed, there is an image below my sample, if you play it with mpv --hwdec=vulkan, it should have the same jagged grid as my image.

Ah, apologies I only checked the output with the NVIDIA sample decoder, which doesn't have the jagged grid here. I also notice your problem with FFmpeg. I'll look into what's going wrong there.

Obegg commented 1 year ago

Edit: Tested today's build with the new patch and the "jagged grid" issue is still an issue.

charlie-ht commented 1 year ago

Edit: Tested today's build with the new patch and the "jagged grid" issue is still an issue.

I checked and this is an issue with FFmpeg's AVC parser, specifically it appears to be parsing scaling lists incorrectly still, they are the biggest difference compared to reference.

philipl commented 1 year ago

There was another patch yesterday to fix h.264 scaling list order. Do you think there is still a bug after that?

philipl commented 1 year ago

FFmpeg/FFmpeg@c070970 may have fixed some H264 streams on Nvidia

This sample still has problems

Just checked again. Looks fine on Intel. Looks wrong on nvidia (535.98). That means it's presumably a driver issue.

AdventurerRussia commented 1 year ago

AMD on Windows appears to be 5000/6000 series according to https://www.amd.com/en/support/kb/release-notes/rn-rad-win-22-11-2-vlk-video-code-decode (possibly 7000 works?). It doesn't work on Polaris: 0.339][e][ffmpeg/video] h264: Device does not support the VK_KHR_video_decode_queue extension!

5300u also writes that there is no support)

Jules-A commented 1 year ago

5300u also writes that there is no support)

I was referring to 5000/6000 series GPUs not CPUs. Your CPU has Vega integrated which was before the 5000 series of GPUs.

FreeTheTech101 commented 1 year ago

NVIDIA released a new driver last night (237.13), but it seems there are still a few kinks to work out:

mpv-shot0001

cyanreg commented 1 year ago

Is that H264?

philipl commented 1 year ago

Does anyone have an h.264 sample that exhibits the blockiness/jaggedness on nvidia - I can't find one locally. There's another scaling list order fix for ffmpeg I need to test out. Note that it's weird that it would work fine on Intel and not nvidia but given that the scaling order thing slipped through in ffmpeg, it might also be wrong inside the Intel driver and that's what it works out.

FreeTheTech101 commented 1 year ago

Does anyone have an h.264 sample that exhibits the blockiness/jaggedness on nvidia - I can't find one locally. There's another scaling list order fix for ffmpeg I need to test out. Note that it's weird that it would work fine on Intel and not nvidia but given that the scaling order thing slipped through in ffmpeg, it might also be wrong inside the Intel driver and that's what it works out.

I can confirm the sample image I posted before was from an H.264 source. Odly, the issue only seems to affect the remux I have from my JP disk, as my ITA one playes perfectly (and both work perfectly while using nvdec). As for sample files, I hope these help (they are < 30 second adverts included with the bonus disk, so I am sure no one will mind).

philipl commented 1 year ago

I can confirm the sample image I posted before was from an H.264 source. Odly, the issue only seems to affect the remux I have from my JP disk, as my ITA one playes perfectly (and both work perfectly while using nvdec). As for sample files, I hope these help (they are < 30 second adverts included with the bonus disk, so I am sure no one will mind).

Bingo. Ben Cheng's zig-zag fix makes this play correctly on nvidia. Although I cannot test immediately, that almost certainly means his fix will break Intel because the driver itself uses the wrong order wrt to the spec.

@cyanreg

alakittty commented 1 year ago

amd 23.8.1 driver(on windows) dropped, can't check it right now to see if vulkan hwdec is working, can anyone else here check please?

cyanreg commented 1 year ago

Just pushed the scaling list fixes to FFmpeg, which should fix H264 on nvidia, and maybe on AMD.

alakittty commented 1 year ago

amd 23.8.1 driver(on windows) dropped, can't check it right now to see if vulkan hwdec is working, can anyone else here check please?

still not working, confirmed