Problems building Torchvision from source

[chinglamchoi]

Hello, I have compiled ffmpeg version 4.2.4 from source and am trying to do python setup.py install after git-cloning to build Torchvision from source (Python 3.8.8, GNU/Linux 5.4.0-58-generic x86_64). However, I'm getting the following error:

cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
g++ -pthread -shared -B /home/user/anaconda3/compiler_compat -L/home/user/anaconda3/lib -Wl,-rpath=/home/user/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/video_reader/video_reader.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/video_stream.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/sync_decoder.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/subtitle_sampler.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/audio_sampler.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/cc_stream.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/audio_stream.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/seekable_buffer.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/memory_buffer.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/subtitle_stream.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/stream.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/decoder.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/time_keeper.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/util.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/decoder/video_sampler.o build/temp.linux-x86_64-3.8/home/user/trial/data/vision/torchvision/csrc/io/video/video.o -L/usr/local/lib -L/usr/local/lib/x86_64-linux-gnu -L/home/user/anaconda3/lib/python3.8/site-packages/torch/lib -lavcodec -lavformat -lavutil -lswresample -lswscale -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-3.8/torchvision/video_reader.so -std=c++14
/home/user/anaconda3/compiler_compat/ld: /usr/local/lib/libavcodec.a(vc1dsp_mmx.o): relocation R_X86_64_PC32 against symbol `ff_pw_9' can not be used when making a shared object; recompile with -fPIC
/home/user/anaconda3/compiler_compat/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1

How do I resolve this issue? Thanks!

[NicolasHug]

I'm not too familiar with this but it seems that the culprit is:

/home/user/anaconda3/compiler_compat/ld: /usr/local/lib/libavcodec.a(vc1dsp_mmx.o): relocation R_X86_64_PC32 against symbol `ff_pw_9' can not be used when making a shared object; recompile with -fPIC

Have you tried to recompile with -fPIC?

[chinglamchoi]

I'm having trouble finding the line of code where libavcodec.a is compiled. I dug through the files in the torchvision source but couldn't find it. Do you know where I can add the -fPIC option?

[NicolasHug]

I believe that this is a flag to pass when you complied ffmpeg from source

[chinglamchoi]

Ah I see, thanks a lot. I used ./configure --enable-shared while compiling ffmpeg from source (due to the static library). However, I still could not compile torchvision from source with ffmpeg support (although ffmpeg was found) and got the error RuntimeError: Not compiled with video_reader support, to enable video_reader support, please install ffmpeg (version 4.2 is currently supported) andbuild torchvision from source. while trying to use io.VideoReader.

I think I will look for an alternate solution for processing video & audio streams in mp4s.

[NicolasHug]

The way torchvision decides whether it should build with ffmpeg support is:

https://github.com/pytorch/vision/blob/d9633fc940faabdd73eb61232011a22de9095ca0/setup.py#L348-L349

After you compiled ffmpeg from source, make sure the executable is discoverable in your $PATH. Perhaps do export PATH=<path of the ffmpeg binary>:$PATH and try compiling torchvision again?

[fmassa]

@chinglamchoi if you are facing issues, you could also try using PyAV for decoding your video and audio streams, and rely on torchvision wrappers around it, which we have in torchvision.io. This will probably be the easiest