dmlc / decord

An efficient video loader for deep learning with smart shuffling that's super easy to digest
Apache License 2.0
1.88k stars 160 forks source link

Error building decord on GPU #186

Open prabhat00155 opened 3 years ago

prabhat00155 commented 3 years ago

I am getting errors when I do make. I've built ffmpeg from source(with GPU support).

[ 21%] Building CXX object CMakeFiles/decord.dir/src/runtime/module_util.cc.o
In file included from /home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/audio_interface.h:9,
                 from /home/prabhatroy/Document/repos/decord/src/audio/audio_reader.h:10,
                 from /home/prabhatroy/Document/repos/decord/src/audio/audio_interface.cc:5:
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:5: error: ‘AVBSFContext’ was not declared in this scope
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
     ^~~~~~~~~~~~
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:5: note: suggested alternative: ‘AVIOContext’
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
     ^~~~~~~~~~~~
     AVIOContext
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:28: error: ‘AVBSFContext’ was not declared in this scope
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                            ^~~~~~~~~~~~
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:28: note: suggested alternative: ‘AVIOContext’
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                            ^~~~~~~~~~~~
                            AVIOContext
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:48: error: ‘av_bsf_free’ was not declared in this scope
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                                                ^~~~~~~~~~~
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:48: note: suggested alternative: ‘av_opt_free’
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                                                ^~~~~~~~~~~
                                                av_opt_free
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:59: error: template argument 1 is invalid
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                                                           ^
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:59: error: template argument 3 is invalid
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:61: error: template argument 1 is invalid
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                                                             ^
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:61: error: template argument 2 is invalid
In file included from /home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/audio_interface.h:9,
                 from /home/prabhatroy/Document/repos/decord/src/audio/audio_reader.h:10,
                 from /home/prabhatroy/Document/repos/decord/src/audio/audio_reader.cc:5:
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:5: error: ‘AVBSFContext’ was not declared in this scope
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
     ^~~~~~~~~~~~
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:5: note: suggested alternative: ‘AVIOContext’
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
     ^~~~~~~~~~~~
     AVIOContext
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:28: error: ‘AVBSFContext’ was not declared in this scope
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                            ^~~~~~~~~~~~
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:28: note: suggested alternative: ‘AVIOContext’
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                            ^~~~~~~~~~~~
                            AVIOContext
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:48: error: ‘av_bsf_free’ was not declared in this scope
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                                                ^~~~~~~~~~~
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:48: note: suggested alternative: ‘av_opt_free’
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                                                ^~~~~~~~~~~
                                                av_opt_free
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:59: error: template argument 1 is invalid
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                                                           ^
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:59: error: template argument 3 is invalid
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:61: error: template argument 1 is invalid
     AVBSFContext, Deleterp<AVBSFContext, void, av_bsf_free> >;
                                                             ^
/home/prabhatroy/Document/repos/decord/src/audio/../../include/decord/../../src/video/ffmpeg/ffmpeg_common.h:187:61: error: template argument 2 is invalid
make[2]: *** [CMakeFiles/decord.dir/build.make:76: CMakeFiles/decord.dir/src/audio/audio_interface.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [CMakeFiles/decord.dir/build.make:90: CMakeFiles/decord.dir/src/audio/audio_reader.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/decord.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

Following is the result of cmake:

(py36_env) [prabhatroy@devgpu003.ftw2 ~/Document/repos/decord/build] cmake .. -DUSE_CUDA=ON -DCMAKE_BUILD_TYPE=Release -DFFMPEG_DIR="/usr/local/bin"
-- The C compiler identification is GNU 8.5.0         
-- The CXX compiler identification is GNU 8.5.0
-- Detecting C compiler ABI info                                                                                                                                                                                                                              
-- Detecting C compiler ABI info - done                        
-- Check for working C compiler: /bin/cc - skipped                                                                             
-- Detecting C compile features                                                                                                                                                                                                                               
-- Detecting C compile features - done                                                                                         
-- Detecting CXX compiler ABI info                         
-- Detecting CXX compiler ABI info - done                      
-- Check for working CXX compiler: /bin/c++ - skipped                                                                                                                                                                                                         
-- Detecting CXX compile features                                                                                              
-- Detecting CXX compile features - done                       
-- The CUDA compiler identification is NVIDIA 11.1.74                                                                                                                                                                                                         
-- Detecting CUDA compiler ABI info                                                                                                                                                                                                                           
-- Detecting CUDA compiler ABI info - done                                                                                     
-- Check for working CUDA compiler: /home/prabhatroy/fbsource/third-party/cuda/cuda_11.1/x64-linux/bin/nvcc - skipped
-- Detecting CUDA compile features                                                                                                                                                                                                                            
-- Detecting CUDA compile features - done                                                                                      
-- Performing Test SUPPORT_CXX11                                                                                               
-- Performing Test SUPPORT_CXX11 - Success                                                                                     
FFMPEG_INCLUDE_DIR = /usr/local/bin/include                                                                                                                                                                                                                   
FFMPEG_LIBRARIES = /usr/local/bin/lib/libavformat.so;/usr/local/bin/lib/libavfilter.so;/usr/local/bin/lib/libavcodec.so;/usr/local/bin/lib/libavutil.so;/usr/local/bin/lib/libavdevice.so;/usr/local/bin/lib/libswresample.so 
-- Looking for pthread.h                                                                                                       
-- Looking for pthread.h - found                                                                                                                                                                                                                              
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD                                                                                     
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed                                                                            
-- Looking for pthread_create in pthreads                                                                                                                                                                                                                     
-- Looking for pthread_create in pthreads - not found                                                                                                                                                                                                         
-- Looking for pthread_create in pthread                       
-- Looking for pthread_create in pthread - found                                                                                                                                                                                                              
-- Found Threads: TRUE                                                                                                                                                                                                                                        
-- Found CUDA_TOOLKIT_ROOT_DIR=/home/prabhatroy/fbsource/third-party/cuda/cuda_11.1/x64-linux
-- Found CUDA_CUDA_LIBRARY=/home/prabhatroy/fbsource/third-party/cuda/cuda_11.1/x64-linux/lib64/stubs/libcuda.so
-- Found CUDA_CUDART_LIBRARY=/home/prabhatroy/fbsource/third-party/cuda/cuda_11.1/x64-linux/lib64/libcudart.so                                                                                                                                                
-- Found CUDA_NVRTC_LIBRARY=/home/prabhatroy/fbsource/third-party/cuda/cuda_11.1/x64-linux/lib/libnvrtc.so                                                                                                                                                    
-- Found CUDA_CUDNN_LIBRARY=CUDA_CUDNN_LIBRARY-NOTFOUND        
-- Found CUDA_CUBLAS_LIBRARY=/home/prabhatroy/fbsource/third-party/cuda/cuda_11.1/x64-linux/lib64/libcublas.so                                                                                                                                                
-- Found CUDA_NVIDIA_ML_LIBRARY=/home/prabhatroy/fbsource/third-party/cuda/cuda_11.1/x64-linux/lib64/stubs/libnvidia-ml.so                                                                                                                                    
-- Found CUDA_NVCUVID_LIBRARY=/usr/lib64/libnvcuvid.so         
-- Build with CUDA support                                 
-- Configuring done                                                                                                                                                                                                                                           
CMake Warning (dev) in CMakeLists.txt:                                                                                                                                                                                                                        
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"                                                                                                                                                                                    
  for policy details.  Use the cmake_policy command to set the policy and                                                                                                                                                                                     
  suppress this warning.                                                                                                                                                                                                                                      

  CUDA_ARCHITECTURES is empty for target "decord".                                                                                                                                                                                                            
This warning is for project developers.  Use -Wno-dev to suppress it.                                                                                                                                                                                         

-- Generating done                                                                                                                                                                                                                                            
-- Build files have been written to: /home/prabhatroy/Document/repos/decord/build             

Versions:

OS: CentOS Stream 8 (x86_64)
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-2)
Clang version: Could not collect
CMake version: version 3.20.2
Libc version: glibc-2.17

Python version: 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.6.13-0_fbk17_5815_gc01d8dbd2635-x86_64-with-centos-8
Is CUDA available: True
CUDA runtime version: 11.1.74
GPU models and configuration: 
GPU 0: Tesla P100-SXM2-16GB
GPU 1: Tesla P100-SXM2-16GB
GPU 2: Tesla P100-SXM2-16GB
GPU 3: Tesla P100-SXM2-16GB
GPU 4: Tesla P100-SXM2-16GB
GPU 5: Tesla P100-SXM2-16GB
GPU 6: Tesla P100-SXM2-16GB
GPU 7: Tesla P100-SXM2-16GB

Nvidia driver version: 470.57.02
ffmpeg version N-104236-g21979cf98e
rhotertj commented 2 years ago

Did you eventually solve it?

prabhat00155 commented 2 years ago

Did you eventually solve it?

No

poke1024 commented 2 years ago

Looks like you are building against ffmpeg 5 instead of ffmpeg 4. Encountered the same issue on macOS (see https://github.com/dmlc/decord/issues/203).

sailordiary commented 2 years ago

In case anyone's interested, here's a git diff showing the changes required to build against the latest FFmpeg (tested):

diff --git a/src/video/ffmpeg/ffmpeg_common.h b/src/video/ffmpeg/ffmpeg_common.h
index b0b973f..f0f7316 100644
--- a/src/video/ffmpeg/ffmpeg_common.h
+++ b/src/video/ffmpeg/ffmpeg_common.h
@@ -21,6 +21,7 @@
 extern "C" {
 #endif
 #include <libavcodec/avcodec.h>
+#include <libavcodec/bsf.h>
 #include <libavformat/avformat.h>
 #include <libavformat/avio.h>
 #include <libavfilter/avfilter.h>
diff --git a/src/video/nvcodec/cuda_threaded_decoder.cc b/src/video/nvcodec/cuda_threaded_decoder.cc
index 62bc7ee..957a90d 100644
--- a/src/video/nvcodec/cuda_threaded_decoder.cc
+++ b/src/video/nvcodec/cuda_threaded_decoder.cc
@@ -17,7 +17,7 @@ namespace decord {
 namespace cuda {
 using namespace runtime;

-CUThreadedDecoder::CUThreadedDecoder(int device_id, AVCodecParameters *codecpar, AVInputFormat *iformat)
+CUThreadedDecoder::CUThreadedDecoder(int device_id, AVCodecParameters *codecpar, const AVInputFormat *iformat)
     : device_id_(device_id), stream_({device_id, false}), device_{}, ctx_{}, parser_{}, decoder_{},
     pkt_queue_{}, frame_queue_{},
     run_(false), frame_count_(0), draining_(false),
@@ -70,7 +70,7 @@ CUThreadedDecoder::CUThreadedDecoder(int device_id, AVCodecParameters *codecpar,
     }
 }

-void CUThreadedDecoder::InitBitStreamFilter(AVCodecParameters *codecpar, AVInputFormat *iformat) {
+void CUThreadedDecoder::InitBitStreamFilter(AVCodecParameters *codecpar, const AVInputFormat *iformat) {
     const char* bsf_name = nullptr;
     if (AV_CODEC_ID_H264 == codecpar->codec_id) {
         // H.264
diff --git a/src/video/nvcodec/cuda_threaded_decoder.h b/src/video/nvcodec/cuda_threaded_decoder.h
index d7e6fcd..61958a1 100644
--- a/src/video/nvcodec/cuda_threaded_decoder.h
+++ b/src/video/nvcodec/cuda_threaded_decoder.h
@@ -46,7 +46,7 @@ class CUThreadedDecoder final : public ThreadedDecoderInterface {
     using FrameOrderQueuePtr = std::unique_ptr<FrameOrderQueue>;

     public:
-        CUThreadedDecoder(int device_id, AVCodecParameters *codecpar, AVInputFormat *iformat);
+        CUThreadedDecoder(int device_id, AVCodecParameters *codecpar, const AVInputFormat *iformat);
         void SetCodecContext(AVCodecContext *dec_ctx, int width = -1, int height = -1, int rotation = 0);
         bool Initialized() const;
         void Start();
@@ -70,7 +70,7 @@ class CUThreadedDecoder final : public ThreadedDecoderInterface {
         void LaunchThreadImpl();
         void RecordInternalError(std::string message);
         void CheckErrorStatus();
-        void InitBitStreamFilter(AVCodecParameters *codecpar, AVInputFormat *iformat);
+        void InitBitStreamFilter(AVCodecParameters *codecpar, const AVInputFormat *iformat);

         int device_id_;
         CUStream stream_;
diff --git a/src/video/video_reader.cc b/src/video/video_reader.cc
index af4858d..99c9635 100644
--- a/src/video/video_reader.cc
+++ b/src/video/video_reader.cc
@@ -145,7 +145,7 @@ VideoReader::~VideoReader(){

 void VideoReader::SetVideoStream(int stream_nb) {
     if (!fmt_ctx_) return;
-    AVCodec *dec;
+    const AVCodec *dec;
     int st_nb = av_find_best_stream(fmt_ctx_.get(), AVMEDIA_TYPE_VIDEO, stream_nb, -1, &dec, 0);
     // LOG(INFO) << "find best stream: " << st_nb;
     CHECK_GE(st_nb, 0) << "ERROR cannot find video stream with wanted index: " << stream_nb;
zhanwenchen commented 1 year ago

To suppress the Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC warning, you can find your SM number/Compute Capability on your GPU (e.g., 3090 Ti is 86) and include it like this:

cmake .. -DUSE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_BUILD_TYPE=Release
zhanwenchen commented 1 year ago

If you don't want to apply the patch yourself or having trouble with it, you can consider using my fork: https://github.com/zhanwenchen/decord

johndpope commented 8 months ago

is the patch for ? it's still referencing the ffmpeg 4. sudo add-apt-repository ppa:savoury1/ffmpeg5

giving it a go with 5 now.

karthik111 commented 4 months ago

I was able to get decord working on my MacBook (arm 64) by installing and linking to FFMPG 4 instead of the default latest version. This removes the header file and library incompatibilities.

% brew install ffmpeg@4

This resulted in a successful install of version 4 with the following additional steps to be done next:

ffmpeg@4 is keg-only, which means it was not symlinked into /opt/homebrew, because this is an alternate version of another formula.

If you need to have ffmpeg@4 first in your PATH, run: echo 'export PATH="/opt/homebrew/opt/ffmpeg@4/bin:$PATH"' >> ~/.zshrc

(if you are using bash, then change above in your bash .profile

For compilers to find ffmpeg@4 you may need to set: export LDFLAGS="-L/opt/homebrew/opt/ffmpeg@4/lib" export CPPFLAGS="-I/opt/homebrew/opt/ffmpeg@4/include"

For pkg-config to find ffmpeg@4 you may need to set: export PKG_CONFIG_PATH="/opt/homebrew/opt/ffmpeg@4/lib/pkgconfig"

% source ~/.zshrc

cd decord (go to the build folder instead your local git repo'ed folder) and remove the old build dir

rm -r build

mkdir build

cd build

% cmake -DFFMPEG=/opt/homebrew/opt/ffmpeg@4 -DFFMPEG_INCLUDE_DIR=/opt/homebrew/opt/ffmpeg@4/include -DFFMPEG_LIBRARY_DIR=/opt/homebrew/opt/ffmpeg@4/lib ..