NVIDIA / GMAT

A toolkit showing GPU's all-round capability in video processing
Other
181 stars 40 forks source link

The format_cuda filter does not work well #5

Closed kdxtb closed 2 years ago

kdxtb commented 2 years ago

Hi, I just wanted to try tensorrt in this framework, but I found that I got full green screen as the final result. The test command is like this: ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i SampleVideo_640x360_1mb.mp4 -vf scale_npp=640:360,format_cuda=rgbpf32le,tensorrt=ESRGAN_x4_723_360.trt,format_cuda=nv12 -c:v h264_nvenc SRResult.mp4

It seams that this is caused by the "format_cuda" filter. I dumped the output of format_cuda filter like this:

ffmpeg -hwaccel cuda  -hwaccel_output_format cuda -i SampleVideo_640x360_1mb.mp4 -vf scale_npp=640:360,format_cuda=rgbpf32le,hwdownload,format=rgbpf32le output.yuv -y
ffmpeg version f41f54b Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --enable-nonfree --enable-libtensorrt --enable-cuda-nvcc --enable-libnpp --enable-libopencv --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --disable-static --enable-shared --enable-pic --nvccflags='-gencode arch=compute_86,code=sm_86'
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'SampleVideo_640x360_1mb.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    creation_time   : 1970-01-01T00:00:00.000000Z
    encoder         : Lavf53.24.2
  Duration: 00:00:06.40, start: 0.000000, bitrate: 1321 kb/s
  Stream #0:0(und): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x368 [SAR 1:1 DAR 40:23], 932 kb/s, 25 fps, 25 tbr, 12800 tbn, 25 tbc (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg4 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, rawvideo, to 'output.yuv':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf58.76.100
  Stream #0:0(und): Video: rawvideo, rgbpf32le(progressive), 640x360 [SAR 45:46 DAR 40:23], q=2-31, 552960 kb/s, 25 fps, 25 tbn (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 rawvideo
frame=  160 fps=0.0 q=-0.0 Lsize=  432000kB time=00:00:06.40 bitrate=552960.0kbits/s speed=  13x    
video:432000kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

I got an output file filled with all 0. But with command that dump nv12 data directly after decode: ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i SampleVideo_640x360_1mb.mp4 -vf scale_npp=640:360,hwdownload,format=nv12 output.yuv -y I could get correct yuv file which can be play by the ffplay.

Can anyone tell me what's the problem? Thanks in advance!

xiaoweiw-nv commented 2 years ago

vf_format_cuda only supports nv12 as input, and your video is in yuv420p, which is not supported at this moment.