dapicard commented 6 years ago

System information (version)

OpenCV => 3.4.1
Operating System / Platform => Ubuntu 16.04 amd64
Compiler => g++ 5.4

Detailed description

I try to use cudacodec VideoReader to read a mkv : cudacodec::createVideoReader("file.mkv")

OpenCV throws this error :

OpenCV(3.4.1) Error: The function/feature is not implemented (The called functionality is disabled for current build or platform) in throw_no_cuda, file /home/damien/code/lib/opencv-3.4.1/opencv-3.4.1/modules/core/include/opencv2/core/private.cuda.hpp, line 111
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(3.4.1) /home/damien/code/lib/opencv-3.4.1/opencv-3.4.1/modules/core/include/opencv2/core/private.cuda.hpp:111: error: (-213) The called functionality is disabled for current build or platform in function throw_no_cuda

I compile OpenCV with external cuda-enabled ffmpeg :

$ ffmpeg
ffmpeg version n3.4.2-6-g1b9b469 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.9) 20160609
  configuration: --enable-cuda --enable-nvenc --enable-cuvid --enable-nonfree --extra-cflags='-I/usr/local/cuda/include -I/home/damien/code/lib/Video_Codec_SDK_8.1.24/Samples/NvCodec/NvEncoder' --extra-ldflags=-L/usr/local/cuda/lib64 --enable-shared --disable-static
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
Hyper fast Audio and Video encoder

I compile OpenCV with this cmake : cmake --enable-shared -D WITH_NVCUVID Giving me this OpenCV configuration :

General configuration for OpenCV 3.4.1 =====================================
  Version control:               unknown

  Extra modules:
    Location (extra):            /home/damien/code/lib/opencv-3.4.1/opencv_contrib-3.4.1/modules
    Version control (extra):     unknown

  Platform:
    Timestamp:                   2018-04-03T15:40:15Z
    Host:                        Linux 4.13.0-37-generic x86_64
...
  Video I/O:
    DC1394:                      YES (ver 2.2.4)
    FFMPEG:                      YES
      avcodec:                   YES (ver 57.107.100)
      avformat:                  YES (ver 57.83.100)
      avutil:                    YES (ver 55.78.100)
      swscale:                   YES (ver 4.8.100)
      avresample:                NO
...
  NVIDIA CUDA:                   YES (ver 9.1, CUFFT CUBLAS)
    NVIDIA GPU arch:             30 35 37 50 52 60 61 70
    NVIDIA PTX archs:

  OpenCL:                        YES (no extra features)
    Include path:                /home/damien/code/lib/opencv-3.4.1/opencv-3.4.1/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load
...

After taking a look at the source code, I see that :

Using cudacodec VideoReader needs the HAVE_NVCUVID envvar to be set : https://github.com/opencv/opencv/blob/3.4.1/modules/cudacodec/src/video_reader.cpp#L49-L54

This envvar is set only if CUDA_nvcuvid_LIBRARY is set : https://github.com/opencv/opencv/blob/3.4.1/cmake/OpenCVDetectCUDA.cmake#L36-L38

And this variable is set only when we are on WIN32 : https://github.com/opencv/opencv/blob/3.4.1/cmake/FindCUDA.cmake#L792-L794

So I think that there is no way to enable NVCUVID. Do I miss something ? If not, I think that it should be fixed.

Furthermore, I don't want to use the CuvidVideoSource, but I want to use the shared FFMPEG lib that already can use cuvid, with the FFmpegVideoSource. And I think that there is no need for OpenCV to link to cuvid to use this ffmpeg backend. So the FFMPEG backend should be available event if HAVE_NVCUVID is not defined. Is it ?

I can do these changes, but I think that having a confirmation before working on could be interresting.

Steps to reproduce

Compile OpenCV 3.4.1 with CUDA 9.1, NVCUVID

try to open a mkv :

cudacodec::createVideoReader("file.mkv")

More informations about this problem : http://answers.opencv.org/question/188525/enable-cudacodecvideoreader/

alalek commented 6 years ago

4847

dapicard commented 6 years ago

This problem seems to be related to #4847, but not exactly the same. (nor the same platform, neither the same problem VideoReader/VideoWriter, not the same OpenCV version).

This doesn't seem to be a duplicate.

alalek commented 6 years ago

8390 (and other duplicate issues).

https://developer.nvidia.com/nvidia-video-codec-sdk:

Note: For Video Codec SDK 7.0 and later, NVCUVID has been renamed to NVDECODE API.

OpenCV has no support for new API and there are no plans to add this. The latest CUDA version with NVCUVID is ~ CUDA 6.5.

Consider using ffmpeg with enabled CUDA features (via normal cv::VideoCapture - but it can't work with CUDA's cv ::GpuMat).

dapicard commented 6 years ago

Thank you !

To use nvdecode with ffmpeg, we have to add parameters like "-hwaccel cuvid -c:v h264_cuvid" ; how can we add these configurations using VideoCapture ? There is no FFMPEG specific properties referenced in https://docs.opencv.org/3.4.1/dc/dfc/group__videoio__flags__others.html.

dapicard commented 6 years ago

I found a way to define the codec used by the FFMpeg backend :

export OPENCV_FFMPEG_CAPTURE_OPTIONS="video_codec|h264_cuvid"

More generally, it is possible to define theses parameters, using the syntax parameter_name|value;parameter_name2|value2

I didn't find an equivalent for the VideoWriter FFMpeg backend.

mister-good-deal commented 5 years ago

So the cudacodec::createVideoReader to read video input directly in a GpuMat is deprecated since CUDA 7 right ?

dapicard commented 5 years ago

Yes, this is actually what I understand. And I've never been able to use cudacodec video reader/writer with CUDA 7+

mister-good-deal commented 5 years ago

Too bad, I'm working a GPU version of a video processing algorithm and the last optimization I could do improve FPS is to read the images directly in the GPU to avoid CPU => GPU upload time (no I cannot use streams and CUDA data / process overlap to reduce this upload time). I'm using CUDA 9.1 and opencv master git branch (v3.4.1).

Maybe dropping OpenCV to use Nvidia video reader to optimize perfs

dapicard commented 5 years ago

I faced the same situation, and this is why I patched the ffmpeg backend to allow to force the codec to use. See https://github.com/opencv/opencv/issues/11480 Compile ffmpeg from sources with CUDA support and compile OpenCV's master using this ffmpeg backend. Then, before starting your OpenCV application, set the environment variable :

OPENCV_FFMPEG_CAPTURE_OPTIONS="video_codec;h264_cuvid"

This way, the video will be decoded using the Nvidia hardware decoder. But it doesn't provide GpuMat directly, so there is always an additional GPU/CPU memory trip...

nicolasMeg commented 5 years ago

I noticed that work has been done on the cuda funcionnalities recently and especially the python wrapper and I was wondering if something is being done on this particular issue or if the workaround suggested by dapicard is the only current solution?

iamsurya commented 5 years ago

This way, the video will be decoded using the Nvidia hardware decoder. But it doesn't provide GpuMat directly, so there is always an additional GPU/CPU memory trip...

Has there been an update to this? I don't see why we still have the videoreader class in the codebase if not. Its been 3 years since NVIDIA made this change but we still don't have a solution to reading to GpuMat directly?

dapicard commented 5 years ago

Has there been an update to this? I don't see why we still have the videoreader class in the codebase if not. Its been 3 years since NVIDIA made this change but we still don't have a solution to reading to GpuMat directly?

It would be nice, indeed but I'm not sure that it would improve significantly the performance : in a project I lead currently, we are using the h264_cuvid in one thread to read the video, and we compute the Mat in some other threads, so the GPU/CPU trips are done alongside of the computing. Furthermore, as well as some algorithms are not implemented on CUDA (typically findContours), we always need to work on a Mat and a GpuMat. So such a functionnality will just save one CPU->GPU trip. On the other hand, rewrite and/or update of the cuda::videoreader to provide a configurable reader could be a bit complicated, so I'm not sure that the benefit justifies the cost. Using ffmpeg backend is a cheap and efficient way to use the Nvidia decoders.

(And if you want to use the nvenc, I did a patch that can't be pulled on the opencv's master for many reasons, but it does the job : https://github.com/dapicard/opencv/tree/3.4.2-nvenc ; you just have to define the FFMPEG encoder to use using the OPENCV_FFMPEG_ENCODER envvar ; once more, cheap and efficient)

iamsurya commented 5 years ago

Thanks. This is great (didn't think about reading the video in a separate thread). I can't find the comment, but there was a note on how opencv would prefer to use OpenCL instead of CUDA because of the proprietary nature of CUDA.

I'm thinking of updating the videoreader sample with a warning that it is currently deprecated and doesn't work, and adding a new sample that reads a video and performs a simple GPU operation (sobel filter?) on the frames instead.

djmv commented 5 years ago

@dapicard how I can run your project to test video reader with GPU ??

dapicard commented 5 years ago

@dapicard how I can run your project to test video reader with GPU ??

You have to use at least OpenCV 3.4.2, then :

Create a videoreader using the FFMPEG backend : cap->open(video_src, CAP_FFMPEG);
set the envvar OPENCV_FFMPEG_CAPTURE_OPTIONS="video_codec;h264_cuvid" This way, the videoreader will use the h264_cuvid codec.

If your source is an rtsp one, add the entry rtsp_transport;tcp ; so the envvar becomes : OPENCV_FFMPEG_CAPTURE_OPTIONS="video_codec;h264_cuvid|rtsp_transport;tcp"

khurramHazen commented 5 years ago

@dapicard Can you share how much a speedup do you get with this cuda FFMPEG trick? I have compiled ffpmeg with cuda support and then used this ffmpeg and cuda to compile opencv. Next I am using your environment variable trick to use cuda acceleration while decoding video files. But I am observing a 2x DROP in performance (simple VideoCapture() vs environment variable & VideoCapture()).

With OPENCV_FFMPEG_CAPTURE_OPTIONS

240.73 FPS
237.46 FPS
235.61 FPS
241.11 FPS
238.27 FPS

Without environment variable

508.61 FPS
527.77 FPS
521.37 FPS
525.50 FPS
522.76 FPS

I am using following code for testing:

import time
import cv2

vid_path = '/input/reencoded.mp4'
cap = cv2.VideoCapture(vid_path)

i = 0
while cap.isOpened():
    if i==0:
        stime = time.time()
    a, frame = cap.read()
    i += 1
    if i==1000:
        etime = time.time()
        s = '{:06.2f} FPS'.format( 1000/(etime-stime) ) 
        print( s )
        # print(frame.shape)
        i = 0

$ ffmpeg
ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --enable-nonfree --enable-shared --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-fPIC --extra-cflags=-I/usr/local/cuda-9.2/include --extra-cflags=-I/usr/local/include --extra-cflags=-I/opt/conda/include --extra-ldflags=-L/usr/local/cuda-9.2/lib64
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

dapicard commented 5 years ago

@khurramHazen I'm not suprise that the performance decreases this way : you are doing, at least, one more memory trip between the GPU memory and the CPU one. And because you're not splitting reading and processing in, at least, two threads, this memory trip could not be done in parellel of other stuff.

When I see some performance increases using h264_cuvid, I was doing a lot of processing operations, some of them on CPU, others on GPU and I split these operations on up to 6 threads, pipelined by blocking and non-blocking queues. And I did it in C++, allowing me to do exactly what I want, when I want. Then, better performance mainly comes from the fact that using h264_cuvid allow to have more CPU cores available for processing stuff as Nvidia cards provides specific "cores" for video read/write. But if you don't use all your CPU cores, you can use them to decode video ; it will be faster.

For a simple video read, I'm pretty sure that you better have to use CPU reader.

khurramHazen commented 5 years ago

@dapicard +1 your reply. If you can share a sample of video reading and/or some processing on it over multiple threads, that'd be REALLY helpful #:1st_place_medal:

stevenyeun commented 5 years ago

@dapicard opencv 3.4.2 vs2017 i did set envvar on windows 10 varname is OPENCV_FFMPEG_CAPTURE_OPTIONS varvalue is video_codec;h264_cuvid

but did cant open avi file.

i saw following msg on console window

[ INFO:0] VIDEOIO: Enabled backends(6, sorted by priority): FFMPEG(1000); MSMF(990); DSHOW(980); VFW(970); CV_IMAGES(960); CV_MJPEG(950)

and VideoCapture open func return not true if (!vcap.open(videoStreamAddress ,CAP_FFMPEG)) { /*cout << "Error opening video stream or file" << endl;*/ cout << "[GOTS_VIDEO_PROC] Open failure" << endl; waitKey(); } please help me i want to decode video with NVIDIA graphic card

dapicard commented 5 years ago

@khurramHazen I think I will push a sample of my processing stream on github in the next days, but I can't promise.

@stevenyeun What happens if you try to open your stream with ffmpeg directly ?

stevenyeun commented 5 years ago

@dapicard i'm sorry I dont know what you said, if i didnt set envvar so i can do open video stream but if i set envvar then i cant open stream.. thank your reply.

dapicard commented 5 years ago

@stevenyeun I've never tested it on Windows, but :

install FFMpeg with CUDA hardware acceleration on Windows (I can't help you for that)
given yout input file/stream : "my_h264_file.mkv", try to do some (useless) stuff using ffmpeg :
- without cuvid : # ffmpeg -i my_h264_file.mkv -c:v libx264 -preset veryslow output.mp4
- with cuvid : # ffmpeg -hwaccel cuvid -c:v h264_cuvid -i my_h264_file.mkv -c:v h264_nvenc output.mp4

If you're not able to use ffmpeg with cuvid in the cli, it will not work with OpenCV.

More information on FFMPEG with cuvid : https://developer.nvidia.com/ffmpeg

stevenyeun commented 5 years ago

@dapicard thanks a lot

dapicard commented 5 years ago

@khurramHazen : as promised, the sample project allowing to process multiple videos in many threads, in C++ : https://github.com/dapicard/opencv-parallelize-example

khurramHazen commented 5 years ago

Thanks @dapicard This will help the people to effectively utilize the HW resources available at their disposal will be a good learning exercise for students like me :)

On Tue, 22 Jan 2019 at 12:26 PM, Damien Picard notifications@github.com wrote:

@khurramHazen https://github.com/khurramHazen : as promised, the sample project allowing to process multiple videos in many threads, in C++ : https://github.com/dapicard/opencv-parallelize-example

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/opencv/opencv/issues/11220#issuecomment-456296771, or mute the thread https://github.com/notifications/unsubscribe-auth/AcGSdVIs62X7JdZQH3oI8zlJt-T-13oXks5vFr0wgaJpZM4TGph2 .

--

--

Khurram Amin / Co-Founder

khurram@hazen.ai sohaib@hazen.ai / +92 346 775 7009 <+966%2055%20578%209103>

hazen.ai http://hazen.ai | smart cameras for smarter cities Startups Center, Next to Gate 2, Umm Al Qura University Makkah Al Mukarramah, Saudi Arabia.

National Incubation Center, LUMS, DHA, Lahore

This e-mail message may contain confidential or legally privileged information and is intended only for the use of the intended recipient(s). Any unauthorized disclosure, dissemination, distribution, copying or the taking of any action in reliance on the information herein is prohibited. E-mails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, or contain viruses. Anyone who communicates with us by e-mail is deemed to have accepted these risks. HAZEN.AI is not responsible for errors or omissions in this message and denies any responsibility for any damage arising from the use of e-mail. Any opinion and other statement contained in this message and any attachment are solely those of the author and do not necessarily represent those of the organization.

wahyubram82 commented 5 years ago

cuda video decoder (nvcuvid), will be deprecated. no longer exists in cuda (latest 10.1)

but if you want to use it with opencv, install NVIDIA Video Decoder SDK https://developer.nvidia.com/nvidia-video-codec-sdk

after install it, you will have nvcuvid.lib in SDK/lib/(platform) directory. just pointing the opencv setting cuda_nvcuvid_library to that file. and make sure you check WITH NVCUVID

A7ocin commented 4 years ago

@wahyubram82 thank you for your suggestion, it has been helpful indeed! By setting the cuda_nvcuvid_library as in the image below and by enabling the flag WITH_NVCUVID, the cudacodec::VideoReader is no more crashing during initialisation (as it was before).

However, it is crashing later while grabbing frames. So my two questions for you at the moment are:

Did you manage to make the cudacodec::VideoReader under Windows?
If so, did you link a custom built version of FFmpeg when compiling OpenCV?

Thank you in advance

abhisbirad commented 4 years ago

Hi,

As the the link : https://github.com/opencv/opencv/issues/10201 they seem to get the video decoder working with CUDA 9.0!.

Is it working with cuda 9.0 and opencv 3.3.1?

Thanks, Abhishek

wahyubram82 commented 4 years ago

@A7ocin : yes I do it in windows, and yes I build ffmpeg, just build it from github https://github.com/FFmpeg/FFmpeg

wahyubram82 commented 4 years ago

@A7ocin: I use opencv not as main tools, I use many tools, for GPU processing, I use the darknet Yolo process, i think, the simple way to reach your purpose in Vision Computing is use the latest method (yolo) and don't re-inventing the wheels. I combine it with opencv for multiple purpose that I need. VideoReader? your mean VideoCapture? I always use VideoCapture function in opencv, It can read video file, play stream video (m3u, etc) or from Webcam Device, make some process with the image recognizer by Yolo and do some marking job with opencv.

I don't say opencv not good but I really like Yolo method, fastest and have good accuracy.

A7ocin commented 4 years ago

@wahyubram82: no wait a second, I mean the cv::cudacodec::VideoReader, not the cv::VideoCapture. Is the cv::cudacodec::VideoReader working for you on your Windows machine? And also, is your custom FFmpeg version somehow linked to the OpenCV build? Thank you

chuanli11 commented 4 years ago

@wahyubram82 did you get the video_reader demo working in Windows?

I managed to run it with OpenCV 3.4.1 + CUDA 10.0 in Ubuntu 18.04 by following these steps: https://github.com/opencv/opencv/issues/10201#issuecomment-402453361

However, couldn't get it working in Windows.

In my Ubuntu case, it seems FFMPEG is not used for the CUDA decoding. I didn't even install it at the beginning until the CPU decoding complained. So I guess FFMPEG is not the problem here.
In my Windows case, I could not properly turn on NVCUVID while building OpenCV. I wonder if you had the same problem: Window: NVIDIA CUDA: YES (ver 10.1, CUFFT CUBLAS FAST_MATH) Ubuntu: NVIDIA CUDA: YES (ver 10.0, CUFFT CUBLAS NVCUVID FAST_MATH)
Just realized my Windows has CUDA 10.1 and my Ubuntu has 10.0, not sure if this would make a difference.

chuanli11 commented 4 years ago

For the record, I got the cuda video_reader demo working on Windows 10 using the native NVidia video codec SDK (no FFMPEG trick as suggested by dapicard). There is no extra trip to transfer data from CPU to GPU for cuda decoding. The original video_reader.cpp also works without modification. Lol

To me the biggest problem was to find the correct versions of a bunch of softwares. I list my settings here. I hope it can help others too:

OS: Window 10 CUDA: 10.0 NVidia-driver: 411.31 (come with the CUDA installation) OpenCV: 4.0.1 Video Codec SDK: Video_Codec_SDK_8.2.16 Visual studio: 2017 community CMake: 3.15.3

What didn't work: OpenCV 3.4.1 with Video_Codec_SDK 8.0, which used dynlink_nvcuvid.h and dynlink_cuviddec.h. That was a big mess.

A few notes:

Copy nvcuvid.h and cuviddec.h from Video_Codec_SDK_8.2.16/Samples/NvCodec/NvDecoder/ to NVIDIA GPU Computing Toolkit/CUDA/v10.0/include
Set WITH_NVCUVID=ON when you build OpenCV from source. Importantly, make sure NVCUVID is included in CMake's NVIDIA CUDA output. Otherwise the "opencv_cudacodec" module can still compile but will miss some implementation. This output works for me: NVIDIA CUDA: YES (ver 10.0, CUFFT CUBLAS NVCUVID FAST_MATH)
In order to have NVCUVID included, I need to set this: CUDA_nvcuvid_LIBRARY=PATH_TO_Video_Codec_SDK_8.2.16/Samples/NvCodec/Lib/x64/nvcuvid.lib
In my case, the original video_camera.cpp (OpenCV 4.0.1) works. GPU decode is significantly faster (a 2070 mobile card is 4x faster than a i7-8750H mobile CPU). You do need to set up the necessary paths in Visual Studio to make the code compile. The most important one is to include the opencv_cudacodec401d.lib as additional library dependency. It has the implementation of the CUDA decoder.

Happy decoding!

A7ocin commented 4 years ago

@chuanli11 thank you, I will try as you suggested and let you know if I manage to make it work. Thank you so much!

A7ocin commented 4 years ago

@chuanli11 that worked flawlessly, thank you! Are you also able to run the video_writer.cpp sample?

byshichen commented 4 years ago

@chuanli11 Thanks for your sharing, have you tested these on Linux platform? And is it possible to use CUDA 10.1 version?

grzegorzk commented 3 years ago

Hi @dapicard , you mentioned (ages ago, I know...)

I didn't find an equivalent for the VideoWriter FFMpeg backend.

Is there any way I can use h264_nvenc encoder when writing frames with VideoWriter? I'm on linux and could not find OPENCV_FFMPEG_WRITER_OPTIONS anywhere in the repository..

dapicard commented 3 years ago

@grzegorzk I did a patch some years ago to manually set the ffmpeg encoder. But it's a fork and, I don't maintain it and it only works with H264, as I remember. So, I can't advise you to use it.

https://github.com/dapicard/opencv/commit/06de2fbe3a4f27cc61be71383f361b967c63cb78

grzegorzk commented 3 years ago

It looks like VideoWriter codec selection change is safe to try it out, I wonder why have it not been merged. There are no merge conflicts so I'll give it a go, thanks!

grzegorzk commented 3 years ago

Checking in to say it works very well. Thanks @dapicard!

JoshuaJakowlew commented 3 years ago

Hi @dapicard , you mentioned (ages ago, I know...)

I didn't find an equivalent for the VideoWriter FFMpeg backend.

Is there any way I can use h264_nvenc encoder when writing frames with VideoWriter? I'm on linux and could not find OPENCV_FFMPEG_WRITER_OPTIONS anywhere in the repository..

This patch would enable h264_nvenc encoding in cv::VideoWriter, not cv::cudacodec::VideoWriter, am I right?

dapicard commented 3 years ago

@JoshuaJakowlew, Yes it does, you're right.

JoshuaJakowlew commented 3 years ago

@JoshuaJakowlew, Yes it does, you're right.

I tried your patch and it doesnt work. Runtime says "Run-Time Check Failure #3 - The variable 'codec' is being used without being initialized." So I inserted codec varible nullptr initialisation and it's working fine, I see GPU load in "Video Encode" tab in Task Manager.

I'm unfamiliar fith github interface, could you please apply the patch for me, or tell how to do it myself?

For other users, who faced the same problem as me

I use vcpkg to build opencv:

./vcpkg install opencv4[ffmpeg]:x64-windows
Go to /pathto/vcpkg/ports/opencv4 and paste this patch.
Then go to profile.cmake and add patch as described here
Now run ./vcpkg remove opencv4[ffmpeg]:x64-windows --recurse
Then again ./vcpkg.exe install opencv4[ffmpeg]:x64-windows You should see your patch listed and applied.
Now create environment variable OPENCV_FFMPEG_ENCODER with h264_nvenc value
Build and run your application, it works for me.

keivanmoazami commented 3 years ago

I download nvidia video sdk from https://developer.nvidia.com/nvidia-video-codec-sdk/download and copy libs & headers into cuda path. then set WITH_NVCUVID=ON and make project. it seems be ok.

Untitled2

csldali commented 3 years ago

@keivanmoazami , can you tell me what libs & headers into cuda path and where is cuda path, is it only copy libs & headers? thanks!

keivanmoazami commented 3 years ago

@keivanmoazami , can you tell me what libs & headers into cuda path and where is cuda path, is it only copy libs & headers? thanks!

copy nvcuvid.lib and nvencodeapi.lib from Video Codec SDK to lib folder of installed cuda path (default: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64). also copy header files from video codec sdk (cuviddec.h, nvcuvid.h, nvEncodeAPI.h) into cuda headers path (default: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\include).

because cmake file search for libs only in cuda path. if you don't find cuda path, you need to install cuda toolkit before build opencv with cuda.

castleguarders commented 3 years ago

Checking in to say it works very well. Thanks @dapicard!

Can you provide information about the configuration it works in? I have not had much success with being able to use h264_nvenc as the encoder. I forward ported the patch to 4.5.1, as well as used dapicards fork as is (4.2).

In both cases the encoder is found, but the encoder open call fails. I have no problems with the decoder. (Another behavior is that if the env for encoder is not set, then there is a segfault).

Fails to open encoder here --> if ((err= avcodec_open2(c, codec, NULL)) < 0) {

Other c++ code examples seem to initialize the context pointer c with the codec (vs the patched code lets it autodetect from the input first). Also avcodec_alloc_context3 is being used in other examples vs avformat_alloc_context..

Cuda 11.2/GTX1650/Ubuntu. Fffmpeg / opencv with cuda enabled. FFmpeg h264_nvenc verified functional.

biel-wangdf3 commented 2 years ago

I download nvidia video sdk from https://developer.nvidia.com/nvidia-video-codec-sdk/download and copy libs & headers into cuda path. then set WITH_NVCUVID=ON and make project. it seems be ok.

according to your method, copy nvidia-video-codec-sdk libs & headers into cuda path, but met errors , can you help me? `opencv_contrib/modules/cudacodec/src/videodecoder.hpp:79:16: error: ‘cuvidDecodePicture’ was not declared in this scope return cuvidDecodePicture(decoder, picParams) == CUDA_SUCCESS; ^~~~~~ /home/wdf/workspace/opencv/opencv_contrib/modules/cudacodec/src/videodecoder.hpp:79:16: note: suggested alternative: ‘tcuvidDecodePicture’ return cuvidDecodePicture(decoder, picParams) == CUDA_SUCCESS; ^~~~~~ tcuvidDecodePicture In file included from /home/wdf/workspace/opencv/opencv_contrib/modules/cudacodec/src/precomp.hpp:57:0, from /home/wdf/workspace/opencv/opencv_contrib/modules/cudacodec/src/video_parser.cpp:44: /home/wdf/workspace/opencv/opencv_contrib/modules/cudacodec/src/video_decoder.hpp: In member function ‘cv::cuda::GpuMat cv::cudacodec::detail::VideoDecoder::mapFrame(int, CUVIDPROCPARAMS&)’: /home/wdf/workspace/opencv/opencv_contrib/modules/cudacodec/src/videodecoder.hpp:87:21: error: ‘cuvidMapVideoFrame’ was not declared in this scope cuSafeCall( cuvidMapVideoFrame(decoder, picIdx, &ptr, &pitch, &videoProcParams) ); ^ /home/wdf/workspace/opencv/opencv/modules/core/include/opencv2/core/private.cuda.hpp:163:61: note: in definition of macro ‘cuSafeCall’

define cuSafeCall(expr) cv::cuda::checkCudaDriverApiError(expr, FILE, LINE, CV_Func)

                                                         ^~~~

[ 55%] Built target opencv_test_imgproc ` @keivanmoazami

opencv / opencv

Unable to enable Cudacodec VideoReader #11220

System information (version)

Detailed description

Steps to reproduce

4847

8390 (and other duplicate issues).

With OPENCV_FFMPEG_CAPTURE_OPTIONS

Without environment variable

define cuSafeCall(expr) cv::cuda::checkCudaDriverApiError(expr, FILE, LINE, CV_Func)