search

GPUOpen-LibrariesAndSDKs / AMF

The Advanced Media Framework (AMF) SDK provides developers with optimal access to AMD devices for multimedia processing
Other
585 stars 149 forks source link

[Bug]: DX11 hardware accelerated transcode fails on FFMPEG v6 #455

Open boyedarat opened 4 months ago

boyedarat commented 4 months ago

Describe the bug Using "-hwaccel d3d11va -hwaccel_output_format d3d11" on FFMPEG for hardware accelerated transcoding fails. But using "-hwaccel dxva2 -hwaccel_output_format dxva2_vld" works just fine. I've tried with many different FFMPEG v6 builds, including latest snapshot releases, same issue.

To Reproduce Steps to reproduce the behavior: Just attempt to transcode any h264 MP4 video to HEVC using AMF on FFMPEG v6, as per your Wiki: https://github.com/GPUOpen-LibrariesAndSDKs/AMF/wiki/FFmpeg%20and%20AMF%20HW%20Acceleration#41-hardware-decode-and-hardware-encode

Setup (please complete the following information):

Debug Log (please upload or paste): No error log, it just exits with a 1KB output file.

D:\Projects>ffmpeg.exe -hwaccel d3d11va -hwaccel_output_format d3d11 -i ..\Videos\Rain_Leak_16-2-2024.mp4 -c:v hevc_amf -c:a libfdk_aac -y Rain_Leak_16-2-2024.mp4

ffmpeg version n6.1.1-ffmpeg-windows-build-helpers Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 10.2.0 (GCC)
  configuration: --pkg-config=pkg-config --pkg-config-flags=--static --extra-version=ffmpeg-windows-build-helpers --enable-version3 --disable-debug --disable-w32threads --arch=x86_64 --target-os=mingw32 --cross-prefix=/home/runner/work/ffmpeg-stable-autobuild/ffmpeg-stable-autobuild/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32- --enable-libcaca --enable-gray --enable-libtesseract --enable-fontconfig --enable-gmp --enable-libass --enable-libbluray --enable-libbs2b --enable-libflite --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libvorbis --enable-libwebp --enable-libzimg --enable-libzvbi --enable-libmysofa --enable-libopenjpeg --enable-libopenh264 --enable-libvmaf --enable-libsrt --enable-libxml2 --enable-opengl --enable-libdav1d --enable-cuda-llvm --enable-gnutls --enable-libsvtav1 --enable-libvpx --enable-libaom --enable-nvenc --enable-nvdec --extra-libs=-lharfbuzz --extra-libs=-lm --extra-libs=-lshlwapi --extra-libs=-lmpg123 --extra-libs=-lpthread --extra-cflags=-DLIBTWOLAME_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCACA_STATIC --enable-amf --enable-libmfx --enable-libaribcaption --enable-gpl --enable-frei0r --enable-librubberband --enable-libvidstab --enable-libx264 --enable-libx265 --enable-avisynth --enable-libaribb24 --enable-libxvid --enable-libdavs2 --enable-libxavs2 --enable-libxavs --extra-cflags='-mtune=generic' --extra-cflags=-O3 --enable-static --disable-shared --prefix=/home/runner/work/ffmpeg-stable-autobuild/ffmpeg-stable-autobuild/sandbox/cross_compilers/mingw-w64-x86_64/x86_64-w64-mingw32 --enable-nonfree --enable-libfdk-aac --enable-decklink
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '..\Videos\Rain_Leak_16-2-2024.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2024-02-16T11:29:15.000000Z
    com.android.version: 14
  Duration: 00:01:47.38, start: 0.000000, bitrate: 17629 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 17346 kb/s, 30 fps, 120 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2024-02-16T11:29:15.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 256 kb/s (default)
    Metadata:
      creation_time   : 2024-02-16T11:29:15.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_amf))
  Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
Output #0, mp4, to 'Rain_Leak_16-2-2024.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    com.android.version: 14
    encoder         : Lavf60.16.100
  Stream #0:0(eng): Video: hevc (hev1 / 0x31766568), d3d11(tv, bt709, progressive), 1920x1080, q=2-31, 2000 kb/s, 120 fps, 15360 tbn (default)
    Metadata:
      creation_time   : 2024-02-16T11:29:15.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 hevc_amf
    Side data:
      displaymatrix: rotation of -0.00 degrees
  Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, s16, 139 kb/s (default)
    Metadata:
      creation_time   : 2024-02-16T11:29:15.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 libfdk_aac
D:\Projects>

Expected behavior

  1. Should behave the same as "ffmpeg.exe -hwaccel dxva2 -hwaccel_output_format dxva2_vld -i ..\Videos\Rain_Leak_16-2-2024.mp4 -c:v hevc_amf -c:a libfdk_aac -y Rain_Leak_16-2-2024.mp4"
  2. A ~28MB video HEVC MP4 output file should have been generated after successful transcoding.

Screenshots N/A.

Additional context Here is the output using DXVA2 with AMF, that works fine:

D:\Projects>ffmpeg.exe -hwaccel dxva2 -hwaccel_output_format dxva2_vld -i ..\Videos\Rain_Leak_16-2-2024.mp4 -c:v hevc_amf -c:a libfdk_aac -y Rain_Leak_16-2-2024.mp4

ffmpeg version n6.1.1-ffmpeg-windows-build-helpers Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 10.2.0 (GCC)
  configuration: --pkg-config=pkg-config --pkg-config-flags=--static --extra-version=ffmpeg-windows-build-helpers --enable-version3 --disable-debug --disable-w32threads --arch=x86_64 --target-os=mingw32 --cross-prefix=/home/runner/work/ffmpeg-stable-autobuild/ffmpeg-stable-autobuild/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32- --enable-libcaca --enable-gray --enable-libtesseract --enable-fontconfig --enable-gmp --enable-libass --enable-libbluray --enable-libbs2b --enable-libflite --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libvorbis --enable-libwebp --enable-libzimg --enable-libzvbi --enable-libmysofa --enable-libopenjpeg --enable-libopenh264 --enable-libvmaf --enable-libsrt --enable-libxml2 --enable-opengl --enable-libdav1d --enable-cuda-llvm --enable-gnutls --enable-libsvtav1 --enable-libvpx --enable-libaom --enable-nvenc --enable-nvdec --extra-libs=-lharfbuzz --extra-libs=-lm --extra-libs=-lshlwapi --extra-libs=-lmpg123 --extra-libs=-lpthread --extra-cflags=-DLIBTWOLAME_STATIC --extra-cflags=-DMODPLUG_STATIC --extra-cflags=-DCACA_STATIC --enable-amf --enable-libmfx --enable-libaribcaption --enable-gpl --enable-frei0r --enable-librubberband --enable-libvidstab --enable-libx264 --enable-libx265 --enable-avisynth --enable-libaribb24 --enable-libxvid --enable-libdavs2 --enable-libxavs2 --enable-libxavs --extra-cflags='-mtune=generic' --extra-cflags=-O3 --enable-static --disable-shared --prefix=/home/runner/work/ffmpeg-stable-autobuild/ffmpeg-stable-autobuild/sandbox/cross_compilers/mingw-w64-x86_64/x86_64-w64-mingw32 --enable-nonfree --enable-libfdk-aac --enable-decklink
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '..\Videos\Rain_Leak_16-2-2024.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2024-02-16T11:29:15.000000Z
    com.android.version: 14
  Duration: 00:01:47.38, start: 0.000000, bitrate: 17629 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 17346 kb/s, 30 fps, 120 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2024-02-16T11:29:15.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 256 kb/s (default)
    Metadata:
      creation_time   : 2024-02-16T11:29:15.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_amf))
  Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
Output #0, mp4, to 'Rain_Leak_16-2-2024.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    com.android.version: 14
    encoder         : Lavf60.16.100
  Stream #0:0(eng): Video: hevc (hev1 / 0x31766568), dxva2_vld(tv, bt709, progressive), 1920x1080, q=2-31, 2000 kb/s, 120 fps, 15360 tbn (default)
    Metadata:
      creation_time   : 2024-02-16T11:29:15.000000Z
      handler_name    : VideoHandle
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 hevc_amf
    Side data:
      displaymatrix: rotation of -0.00 degrees
  Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, s16, 139 kb/s (default)
    Metadata:
      creation_time   : 2024-02-16T11:29:15.000000Z
      handler_name    : SoundHandle
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 libfdk_aac
[vost#0:0/hevc_amf @ 0000021aea4eadc0] More than 1000 frames duplicated5.1kbits/s dup=956 drop=0 speed=2.66x
[out#0/mp4 @ 0000021aea30eb00] video:26199kB audio:1831kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.581092%
frame=12884 fps=310 q=-0.0 Lsize=   28193kB time=00:01:47.35 bitrate=2151.3kbits/s dup=9663 drop=0 speed=2.58x
D:\Projects>
MikhailAMD commented 4 months ago

We are aware of a regression in the latest public driver: incompatibility between FFmpeg use of D3D11VA and AMF encoder, if no filters are in-between. The fix is ready and will appear in one of future driver packages. We will inform when it is available. For now please use one of older drivers : 23.12.1

boyedarat commented 4 months ago

We are aware of a regression in the latest public driver: incompatibility between FFmpeg use of D3D11VA and AMF encoder, if no filters are in-between. The fix is ready and will appear in one of future driver packages. We will inform when it is available. For now please use one of older drivers : 23.12.1

Thanks for the confirmation. Please keep this issue open. I'll test again when the new driver package is out and update the status here.

EmmettBrownn commented 4 months ago

I've just seen that new drivers have been released. Have you had a chance to try them out ?

boyedarat commented 4 months ago

@MikhailAMD Just tested with Adrenaline 24.2.1 on the same system. FFMPEG now just hangs.

On 24.1.1, it would just exit, with a 1KB output file.

MikhailAMD commented 4 months ago

Yes, the fix missed 24.2.1. I am tracking and will let know.

boyedarat commented 3 months ago

Yes, the fix missed 24.2.1. I am tracking and will let know.

@MikhailAMD Adrenaline 24.3.1 is now out and guess what? It is still not fixed. Is AMD planning to actually fix this issue? Or should we just give up?

MikhailAMD commented 3 months ago

I tried myself: reproduced the problem with 24.1.1 driver and after installing 24.3.1 the issue is gone. One thing to check - you have two GPUs: one discreet and one from Ryzen. Do you know which one is used by default? Can you experiment with "-gpu 0" and "-gpu 1" in ffmpeg CMD?

boyedarat commented 3 months ago

@MikhailAMD The options '-gpu 0' or '-gpu 1' don't seem to be AMF options, but rather for NVENC. Here is the full command line I use for FFMPEG:

  1. Using the Radeon 680M on the Ryzen 9 6900HX: ffmpeg.exe -hwaccel d3d11va -hwaccel_output_format d3d11 -hwaccel_device 0 -i video1.mp4 -c:v hevc_amf -usage high_quality -c:a libfdk_aac -y video2.mp4
  2. Using the discrete Radeon RX 6850M XT: ffmpeg.exe -hwaccel d3d11va -hwaccel_output_format d3d11 -hwaccel_device 1 -i video1.mp4 -c:v hevc_amf -usage high_quality -c:a libfdk_aac -y video2.mp4

Both of the above options cause the FFMPEG process to freeze. I have to kill the process to exit it. This is the same FFMPEG build I posted in my original bug report above.

If you have a FFMPEG build or options that work for you. Please post it here. Thanks.

MikhailAMD commented 3 months ago

I used FFmpeg 6.1.1 full release build from here: https://www.gyan.dev/ffmpeg/builds/ My command line was the same as the original one. AMF in this command line will get D3D11 device from HW context so you are right that "-hwaccel_device" should be used to select the GPU device. Before updating drivers I usually uninstall all AMD drivers using the cleanup utility to ensure fresh install: https://www.amd.com/en/support/kb/faq/gpu-601 Also, please enable verbose FFmpeg log. The problem you see may be somewhat different from the fixed one. Your latest command line includes "-usage high_quality" which would enable PA. PA requires more frames to be accumulated before encoder can produce a compressed frame. The defaults may cause deadlock. I suggest to try without this usage and read this article for PA: https://github.com/GPUOpen-LibrariesAndSDKs/AMF/blob/master/amf/doc/AMF_Video_PreAnalysis_API.md. You may need to add more frames by "-extra_hw_frames". Another problem could be that D3D11VA allocates output frames without D3D11_BIND_SHADER_RESOURCE flag. But AMF PA needed it as it is using D3D11. We plan to introduce a patch to FFmpeg for this. If you want to use PA you may need to enable SW decoding for now.

boyedarat commented 3 months ago

We plan to introduce a patch to FFmpeg for this. If you want to use PA you may need to enable SW decoding for now.

I've managed it to get to work with these options: ffmpeg.exe -threads 8 -hwaccel d3d11va -hwaccel_output_format d3d11 -hwaccel_device 1 -i video1.mp4 -c:v hevc_amf -rc hqvbr -c:a libfdk_aac -y video2.mp4

Not being able to use -usage high_quality is a huge bummer for me. Also, dxva2 is not available on my dedicated Radeon RX 6850M XT, only d3d11 is an option for hardware transcoding.

I am not sure how far along AMD has gotten along with FFPMEG 7 support, now that it is out. It is really frustrating that support on such a popular tool is just so poor.

Plootie commented 3 weeks ago

Any update on this? Running to FFMPEG freezes with PA enabled from a fresh driver install and newly built FFMEPG with AMF

MikhailAMD commented 3 weeks ago

There are two separate issues here:

  1. PA fails with D3D11VA decoding enabled. Reason: decoder output texture allocation needs to add D3D11_BIND_SHADER_RESOURCE flag to be able to run PA shaders. We discussed this with FFmpeg, they don't like adding this flag unconditionally, command line parameter could be an option but FFmpeg HW contexts don't have own parameter table so it is not a small change. We plan to add this.
  2. Add AMF HW context, AMF decoder and other components to FFmpeg. We submitted two sets of patches already, got number of comments and preparing the new one. You can find them in FFmpeg patchwork.