rsnk96 / Ubuntu-Setup-Scripts

Scripts to help you set up your Ubuntu quickly, especially if you're in any subfield of Data Science or AI!
Apache License 2.0
128 stars 76 forks source link

Linking error while installing OpenCV #52

Closed rajat2004 closed 4 years ago

rajat2004 commented 4 years ago

An error which I encountered while running opencvDirectInstall.sh. This was done on a fresh installation of Ubuntu 18.04 with both the 1st and 2nd scripts used before this.

In the 1-BasicSetup.sh, Anaconda installation was commented out, nvidia-drivers were enabled. No errors in running it and ZSH was working as expected. No changes in 2, everything ran smoothly. Cuda 10.0 was installed before the OpenCV installation. Reboots between running each script and Cuda.

Error -

/usr/bin/ld: warning: libswresample.so.3, needed by //opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libvpx.so.6, needed by //opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libaom.so.0, needed by //opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libfdk-aac.so.2, needed by //opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libx265.so.179, needed by //opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58, not found (try using -rpath or -rpath-link)
//opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58: undefined reference to `aom_codec_enc_config_default'
//opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58: undefined reference to `aacDecoder_DecodeFrame'
//opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58: undefined reference to `aacDecoder_AncDataInit'
//opt/ffmpeg-build-script/workspace/lib/libavcodec.so.58: undefined reference to `vpx_codec_get_cx_data'
...

FFmpeg building and installation worked, but after going back and looking through the output, saw this -

Adding ffmpeg's libraries to LD_LIBRARY_PATH
/home/rajat/.zim/init.zsh: line 5: autoload: command not found
/home/rajat/.zim/init.zsh: line 11:  ! ${+ZIM_HOME} : bad substitution
/home/rajat/.zim/init.zsh: line 20: bindkey: command not found
/home/rajat/.zim/init.zsh: line 24: syntax error near unexpected token `)'
/home/rajat/.zim/init.zsh: line 24: `() {'
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 35:  ! ${+ZSH_AUTOSUGGEST_HIGHLIGHT_STYLE} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 38:  ! ${+ZSH_AUTOSUGGEST_ORIGINAL_WIDGET_PREFIX} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 42:  ! ${+ZSH_AUTOSUGGEST_STRATEGY} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 45:  ! ${+ZSH_AUTOSUGGEST_CLEAR_WIDGETS} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 48:  ! ${+ZSH_AUTOSUGGEST_ACCEPT_WIDGETS} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 51:  ! ${+ZSH_AUTOSUGGEST_EXECUTE_WIDGETS} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 54:  ! ${+ZSH_AUTOSUGGEST_PARTIAL_ACCEPT_WIDGETS} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 57:  ! ${+ZSH_AUTOSUGGEST_IGNORE_WIDGETS} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 60:  ! ${+ZSH_AUTOSUGGEST_COMPLETIONS_PTY_NAME} : bad substitution
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 95: syntax error near unexpected token `('
/opt/.zsh/zsh-autosuggestions/zsh-autosuggestions.zsh: line 95: `       user:_zsh_autosuggest_(bound|orig)_*)'

Ok, so for some reason, zsh-autosuggestions was loading, which was pretty strange in itself since it was working correctly till then. Due to this, the LD_LIBRARY_PATH and the associated ones didn't get loaded. Opening a new terminal, it was loading correctly and the new FFmpeg paths were also set.

So started a new terminal, checked the paths, cleared OpenCV build and ran it again, still the same problem. Reboot, same thing. Was getting quite frustrated by now. FFmpeg was working.

Now what I did that fixed it (Some more things that I tried, but these seem to be relevant)-

export LD_LIBRARY_PATH=/opt/ffmpeg-build-script/workspace/lib/:$LD_LIBRARY_PATH

sudo sh -c 'echo "/opt/ffmpeg-build-script/workspace/lib" > /etc/ld.so.conf.d/opencv.conf'
sudo ldconfig

Will try to reproduce it, and to figure out what to do to prevent it

One idea from the ZSH error - Export the required PATHS and then echo them to the SHELLRC, rather than to source the SHELLRC, it's an inelegant solution, but should work.

rsnk96 commented 4 years ago

Interesting

And on the contrary, I would say exporting the variable into the current environment is more elegant than reloading the environment as a whole.

Will try to reproduce in a docker container and check if this works

rajat2004 commented 4 years ago

And on the contrary, I would say exporting the variable into the current environment is more elegant than reloading the environment as a whole.

On second thought, yeah, that's true. I'll mostly soon open a PR for this

Will try to reproduce in a docker container and check if this works

Great. thanks! I'll also try, though will take some time.

rsnk96 commented 4 years ago

Fixed via #53

rajat2004 commented 4 years ago

I'm reopening the issue since the problem is still occurring with the latest code changes (Another user, not mine) Screenshot from 2020-01-01 10-18-56

Some more problems also occurred, with libegl.so (though this possibly was a manual installation issue with Nvidia drivers),libz (caused by Anaconda) Update: I've tried to recreate the problem in Travis with some success, branch - https://github.com/rajat2004/Ubuntu-Setup-Scripts/tree/test-opencv Travis build - https://travis-ci.com/rajat2004/Ubuntu-Setup-Scripts/jobs/271557371 It builds and installs, but the warning is present

WhatsApp Image 2020-01-01 at 12 29 13 PM WhatsApp Image 2020-01-01 at 10 36 03 AM

and a checkinstall packaging problem WhatsApp Image 2020-01-01 at 12 28 15 PM Googling this shows it happens sometimes with Java due to some reason, OpenCV builds Java wrappers by default Finally did a make install so as to not go through the whole thing again

rajat2004 commented 4 years ago

Since OpenCV throws errors if it's not able to find the FFmpeg libs, isn't it already using the compiled libraries rather than the system ones (First thing mentioned in https://github.com/rsnk96/Ubuntu-Setup-Scripts/issues/20) The example of zlib given there isn't actually being compiled from source, no such flags are being passed to it during the configuration

rajat2004 commented 4 years ago

Fixed in https://github.com/rsnk96/Ubuntu-Setup-Scripts/pull/56