search

abhiTronix / deffcode

A cross-platform High-performance FFmpeg based Real-time Video Frames Decoder in Pure Python 🎞️⚡
https://abhitronix.github.io/deffcode
Apache License 2.0
178 stars 3 forks source link

[Question]: get first frame is too slow 6000ms #36

Closed wuqingzhou828 closed 1 year ago

wuqingzhou828 commented 1 year ago

Issue guidelines

Issue Checklist

Describe your Question

rtmp url ffmpeg -re -stream_loop -1 -vsync 0 -i ./video/demo_big.mp4 -c:v copy -preset ultrafast -f flv rtmp://ip:1935/live/demobig

I use alqutami/rtmp-hls:latest-alpine (docker image) as rtmp server

when I run my python code, FFdecoder get the fisrt frame need more than 5800 ms

Terminal log output(Optional)

No response

Python Code(Optional)

from deffcode import FFdecoder
import time

# define suitable FFmpeg parameter
ffparams = {
    "-vcodec": "h264_cuvid",  # skip any decoder and let FFmpeg chose
    "-ffprefixes": [
        "-vsync",
        "0",
        "-hwaccel", # chooses appropriate HW accelerator
        "cuda",
        "-hwaccel_output_format", # keeps the decoded frames in GPU memory
        "cuda",
    ],
    "-custom_resolution": "null",  # discard `-custom_resolution`
    "-framerate": "null",  # discard `-framerate`
    "-vf": "scale_npp=format=yuv420p,hwdownload,format=yuv420p,fps=25.0",  # define your filters
}

# initialize and formulate the decoder with params and custom filters
decoder = FFdecoder(
    "rtmp://ip:1935/live/demobig",
    frame_format='bgr24', verbose=True, **ffparams  # discard frame_format
).formulate()

i = 1
while True:
    t1 = time.time()
    frame = next(decoder.generateFrame())
    t2 = time.time()
    print(i, (t2 - t1) * 1000, 'ms')
    i += 1
decoder.terminate()

DeFFcode Version

0.2.4

Python version

3.8

Operating System version

nvcr.io/nvidia/tensorrt:22.04-py3 and ffmpeg gpu

Any other Relevant Information?

No response

abhiTronix commented 1 year ago

frame_format='bgr24',

Remove this. This won't work for Hardware decoded frame.

"-vcodec": "h264_cuvid",

Change this to "-vcodec": None

And I advise you to not use hardware decoding at all for Network streams. Can you paste the output from terminal while running this code?

wuqingzhou828 commented 1 year ago

after i set "-vcodec": None and frame_format=None, I get this error:Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:1' and the filter 'auto_scaler_0' and my log like this 06:06:09 :: Utilities :: INFO :: Running DeFFcode Version: 0.2.4 06:06:09 :: FFhelper :: DEBUG :: Final FFmpeg Path: ffmpeg 06:06:09 :: FFhelper :: DEBUG :: FFmpeg validity Test Passed! 06:06:09 :: FFhelper :: DEBUG :: Found valid FFmpeg Version: b'4.3.4' installed on this system 06:06:09 :: Sourcer :: DEBUG :: Found valid FFmpeg executable: ffmpeg. 06:06:10 :: FFhelper :: WARNING :: Source isn't a valid Image Sequence 06:06:10 :: FFhelper :: DEBUG :: URL scheme rtmp is supported by FFmpeg. 06:06:17 :: Sourcer :: DEBUG :: Extracting Metadata... 06:06:17 :: Sourcer :: DEBUG :: Metadata Extraction completed successfully! 06:06:17 :: FFdecoder :: INFO :: Using default rgb24 pixel-format for this pipeline. 06:06:17 :: FFdecoder :: CRITICAL :: Manually discarding -size/-s FFmpeg parameter from this pipeline. 06:06:17 :: FFdecoder :: INFO :: Default source resolution will be used for this pipeline for defining output resolution. 06:06:17 :: FFdecoder :: CRITICAL :: Manually disabling -framerate/-r FFmpeg parameter for this pipeline. 06:06:17 :: FFdecoder :: INFO :: Default source framerate will be used for this pipeline for defining output framerate. 06:06:17 :: FFdecoder :: INFO :: Live/Network Stream detected! Number of frames in given source are not known. 06:06:17 :: FFdecoder :: CRITICAL :: Activating Video-Only Mode of Operation. 06:06:17 :: FFdecoder :: DEBUG :: Executing FFmpeg command: ffmpeg -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i rtmp://192.168.35.230:1935/live/demobig -pix_fmt rgb24 -vf scale_npp=format=yuv420p,hwdownload,format=yuv420p,fps=25.0 -f rawvideo - ffmpeg version 4.3.4 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1) configuration: --prefix=/usr/local/ffmpeg --enable-shared --enable-nonfree --enable-gpl --enable-version3 --enable-libmp3lame --enable-libvpx --enable-libopus --enable-opencl --enable-libxcb --enable-opengl --enable-nvenc --enable-vaapi --enable-vdpau --enable-ffplay --enable-ffprobe --enable-libxvid --enable-libnpp --enable-libx264 --enable-libx265 --enable-openal --enable-openssl --enable-cuda-nvcc --enable-cuvid --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 libavutil 56. 51.100 / 56. 51.100 libavcodec 58. 91.100 / 58. 91.100 libavformat 58. 45.100 / 58. 45.100 libavdevice 58. 10.100 / 58. 10.100 libavfilter 7. 85.100 / 7. 85.100 libswscale 5. 7.100 / 5. 7.100 libswresample 3. 7.100 / 3. 7.100 libpostproc 55. 7.100 / 55. 7.100 Input #0, flv, from 'rtmp://ip:1935/live/demobig': Metadata: Server : NGINX RTMP (github.com/arut/nginx-rtmp-module) displayWidth : 1920 displayHeight : 1080 fps : 25 profile : level : Duration: 00:00:00.00, start: 649.280000, bitrate: N/A Stream #0:0: Data: none Stream #0:1: Video: h264 (High), yuv420p(progressive), 1920x1080, 25 fps, 25 tbr, 1k tbn, 50 tbc Stream mapping: Stream #0:1 -> #0:0 (h264 (native) -> rawvideo (native)) Press [q] to stop, [?] for help [h264 @ 0x55b62edc84c0] decoder->cvdl->cuvidCreateDecoder(&decoder->decoder, params) failed -> CUDA_ERROR_INVALID_VALUE: invalid argument [h264 @ 0x55b62edc84c0] Using more than 32 (35) decode surfaces might cause nvdec to fail. [h264 @ 0x55b62edc84c0] Try lowering the amount of threads. Using 16 right now. [h264 @ 0x55b62edc84c0] Failed setup for format cuda: hwaccel initialisation returned error. Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:1' and the filter 'auto_scaler_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented Error while processing the decoded data for stream #0:1 Conversion failed!

abhiTronix commented 1 year ago

-pix_fmt rgb24

Means you didn't put frame_format="null".

wuqingzhou828 commented 1 year ago

-pix_fmt rgb24

Means you didn't put frame_format="null".

sorry, i just put frame_format=None after i put frame_format='null', i get this error: [h264 @ 0x563135bebf80] decoder->cvdl->cuvidCreateDecoder(&decoder->decoder, params) failed -> CUDA_ERROR_INVALID_VALUE: invalid argument [h264 @ 0x563135bebf80] Using more than 32 (35) decode surfaces might cause nvdec to fail. [h264 @ 0x563135bebf80] Try lowering the amount of threads. Using 16 right now. [h264 @ 0x563135bebf80] Failed setup for format cuda: hwaccel initialisation returned error. Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:1' and the filter 'auto_scaler_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented Error while processing the decoded data for stream #0:1 Conversion failed! Traceback (most recent call last): frame = next(decoder.generateFrame()) StopIteration

This is my code ` from deffcode import FFdecoder import time from vidgear.gears import WriteGear

define suitable FFmpeg parameter

ffparams = { "-vcodec": None, # skip any decoder and let FFmpeg chose "-ffprefixes": [ "-vsync", "0", "-hwaccel", # chooses appropriate HW accelerator "cuda", "-hwaccel_output_format", # keeps the decoded frames in GPU memory "cuda", ], "-custom_resolution": "null", # discard -custom_resolution "-framerate": "null", # discard -framerate "-vf": "scale_npp=format=yuv420p,hwdownload,format=yuv420p,fps=25.0", # define your filters }

initialize and formulate the decoder with params and custom filters

decoder = FFdecoder( "rtmp://ip:1935/live/demobig", frame_format='null', verbose=True, *ffparams # discard frame_format ).formulate() i = 1 while True: t1 = time.time() frame = next(decoder.generateFrame()) t2 = time.time() print(i, (t2 - t1) 1000, 'ms') i += 1 decoder.terminate() This is my log 02:54:58 :: Utilities :: INFO :: Running DeFFcode Version: 0.2.4 02:54:58 :: Helper :: INFO :: Running VidGear Version: 0.2.6 02:54:58 :: FFhelper :: DEBUG :: Final FFmpeg Path: ffmpeg 02:54:58 :: FFhelper :: DEBUG :: FFmpeg validity Test Passed! 02:54:58 :: FFhelper :: DEBUG :: Found valid FFmpeg Version:b'4.3.4'installed on this system 02:54:58 :: Sourcer :: DEBUG :: Found valid FFmpeg executable:ffmpeg. 02:54:58 :: FFhelper :: WARNING :: Source isn't a valid Image Sequence 02:54:58 :: FFhelper :: DEBUG :: URL schemertmpis supported by FFmpeg. 02:55:06 :: Sourcer :: DEBUG :: Extracting Metadata... 02:55:06 :: Sourcer :: DEBUG :: Metadata Extraction completed successfully! 02:55:06 :: FFdecoder :: CRITICAL :: Manually discardingframe_format(or-pix_fmt) parameter from this pipeline. 02:55:06 :: FFdecoder :: WARNING :: No usable pixel-format defined. Switching to defaultyuv420ppixel-format! 02:55:06 :: FFdecoder :: CRITICAL :: Manually discarding-size/-sFFmpeg parameter from this pipeline. 02:55:06 :: FFdecoder :: INFO :: Default source resolution will be used for this pipeline for defining output resolution. 02:55:06 :: FFdecoder :: CRITICAL :: Manually disabling-framerate/-rFFmpeg parameter for this pipeline. 02:55:06 :: FFdecoder :: INFO :: Default source framerate will be used for this pipeline for defining output framerate. 02:55:06 :: FFdecoder :: INFO :: Live/Network Stream detected! Number of frames in given source are not known. 02:55:06 :: FFdecoder :: CRITICAL :: Activating Video-Only Mode of Operation. 02:55:06 :: FFdecoder :: DEBUG :: Executing FFmpeg command:ffmpeg -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i rtmp://ip:1935/live/demobig -vf scale_npp=format=yuv420p,hwdownload,format=yuv420p,fps=25.0 -f rawvideo -` ffmpeg version 4.3.4 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1) configuration: --prefix=/usr/local/ffmpeg --enable-shared --enable-nonfree --enable-gpl --enable-version3 --enable-libmp3lame --enable-libvpx --enable-libopus --enable-opencl --enable-libxcb --enable-opengl --enable-nvenc --enable-vaapi --enable-vdpau --enable-ffplay --enable-ffprobe --enable-libxvid --enable-libnpp --enable-libx264 --enable-libx265 --enable-openal --enable-openssl --enable-cuda-nvcc --enable-cuvid --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 libavutil 56. 51.100 / 56. 51.100 libavcodec 58. 91.100 / 58. 91.100 libavformat 58. 45.100 / 58. 45.100 libavdevice 58. 10.100 / 58. 10.100 libavfilter 7. 85.100 / 7. 85.100 libswscale 5. 7.100 / 5. 7.100 libswresample 3. 7.100 / 3. 7.100 libpostproc 55. 7.100 / 55. 7.100 Input #0, flv, from 'rtmp://ip:1935/live/demobig': Metadata: Server : NGINX RTMP (github.com/arut/nginx-rtmp-module) displayWidth : 1920 displayHeight : 1080 fps : 25 profile : level : Duration: 00:00:00.00, start: 2565.120000, bitrate: N/A Stream #0:0: Data: none Stream #0:1: Video: h264 (High), yuv420p(progressive), 1920x1080, 25 fps, 25 tbr, 1k tbn, 50 tbc Stream mapping: Stream #0:1 -> #0:0 (h264 (native) -> rawvideo (native)) Press [q] to stop, [?] for help [h264 @ 0x563135bebf80] decoder->cvdl->cuvidCreateDecoder(&decoder->decoder, params) failed -> CUDA_ERROR_INVALID_VALUE: invalid argument [h264 @ 0x563135bebf80] Using more than 32 (35) decode surfaces might cause nvdec to fail. [h264 @ 0x563135bebf80] Try lowering the amount of threads. Using 16 right now. [h264 @ 0x563135bebf80] Failed setup for format cuda: hwaccel initialisation returned error. Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:1' and the filter 'auto_scaler_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented Error while processing the decoded data for stream #0:1 Conversion failed! Traceback (most recent call last): frame = next(decoder.generateFrame()) StopIteration

abhiTronix commented 1 year ago

@wuqingzhou828 I don't think the RTSP source provided by you is compatible with CUDA and anyways you can't use the output frames for any video processing other than writing to disk, because yuv24 is the only pixel format. So I don't think this is an issue. Just remove CUDA code from ffparams dictionary and it will work without any lag.