search

NVIDIA / VideoProcessingFramework

Set of Python bindings to C++ libraries which provides full HW acceleration for video decoding, encoding and GPU-accelerated color space and pixel format conversions
Apache License 2.0
1.32k stars 233 forks source link

Is C++ supported by VPF? #524

Closed ithanwu closed 1 year ago

ithanwu commented 1 year ago

I'm developing a project in C++ ,but when I cloned this repo ,I found No C++ sample codes In my satuation , I need to decode multiple rtsp streams ,and then analyze them by my DNN modle ,after which I would draw my result on the video frame and then encode them to a rtmp stream to show it to people. So,is it possible to use this in C++ , I know ,usually in c++ developer should use a gpu enabled ffmpeg to decode or encode video stream,But As you know , It's fucking complex to use that,And my project gives me limited time,Is there any solotion to use VPF in my C++ project ? Looking forward your reply! Many many thx!

theHamsta commented 1 year ago

While there are no samples it is certainly possible to use VPF from C++. There are two libraries TC and PyNvCodec. TC (https://github.com/NVIDIA/VideoProcessingFramework/tree/master/src/TC, link against target TC) can be used directly from a CMake C++ project, while PyNvCodec is directly compiled into a Python module and would require modifications to VPFs build system to use directly (this was requested in another issue). PyNvCodec contains some extra abstractions and would allow you to use the same API in C++ as you would in Python.

For alternatives to FFmpeg, I can recommend the sample classes of the Video Codec SDK https://developer.nvidia.com/nvidia-video-codec-sdk/download (in path Samples/NvCodec/{NvDecoder,NvEncoder}, don't use the button "Source Code" but "Download now"). They are pretty high-level and MIT licensed.

To also handle the rtmp part of your pipeline it would also be possible to use a GStreamer/DeepStream pipeline in case you are running on Linux (CUDA-enabled GStreamer also runs on Windows but you need to build it from source). You can use both GPU-decoders/encoders from DeepStream or from GStreamer and build a GPU enabled pipeline. You can try first on command line and then use the same pipelines in your app (gst_parse_launch). If you want to use the data in your app you can use appsrc/appsink to usebuffers produced by your pipeline in your app. If those buffers are x-video/raw(memory:NVVM) or x-video/raw(memory:CUDA) then those buffers will not contain the raw video data but you must cast them to structs describing the data layout on GPU. For `x-video/raw the data is already on CPU and can be accessed directly.

ithanwu commented 1 year ago

thanks for your fast response