search

MTG / essentia

C++ library for audio and music analysis, description and synthesis, including Python bindings
http://essentia.upf.edu
GNU Affero General Public License v3.0
2.85k stars 533 forks source link

Cannot build with gaia and examples #972

Open halaei opened 4 years ago

halaei commented 4 years ago

I am trying to build on Ubunbu 18.04 but it fails. Here is a script to reproduce:

# clone
git clone git@github.com:MTG/essentia.git # git checkout 85ebde3b47da43ea09fe7c936da6b2f7cdf04212
git clone git@github.com:MTG/gaia.git # git checkout 875a8c1eb4b88864d8f904dcd8622b3c421993bc
# depencendies
sudo apt-get install build-essential libqt4-dev libyaml-dev swig python-dev pkg-config libeigen3-dev
sudo apt-get install build-essential libeigen3-dev libyaml-dev libfftw3-dev libavcodec-dev libavformat-dev libavutil-dev libavresample-dev python-dev libsamplerate0-dev libtag1-dev libchromaprint-dev python-six
sudo apt-get install python3-dev python3-numpy-dev python3-numpy python3-yaml
# 1. install gaia
cd gaia
./waf configure
./waf
sudo ./waf install
# 2. install essentia
cd ../essentia/
./waf configure --with-vamp --with-gaia --with-examples --with-static-examples --with-example=streaming_extractor_music,streaming_extractor_freesound,streaming_extractor_music_svm
./waf

The last ./waf will fail with the following message:

[311/423] Linking build/src/examples/essentia_standard_onsetrate
[312/423] Linking build/src/examples/essentia_standard_beatsmarker
[313/423] Linking build/src/examples/essentia_standard_mfcc
[314/423] Linking build/src/examples/essentia_standard_rhythmtransform
[315/423] Linking build/src/examples/essentia_standard_fadedetection
/usr/bin/ld: cannot find -lfdk-aac
collect2: error: ld returned 1 exit status

/usr/bin/ld: cannot find -lfdk-aac
collect2: error: ld returned 1 exit status

/usr/bin/ld: cannot find -lfdk-aac
collect2: error: ld returned 1 exit status

/usr/bin/ld: cannot find -lfdk-aac
collect2: error: ld returned 1 exit status

/usr/bin/ld: cannot find -lfdk-aac
collect2: error: ld returned 1 exit status

Waf: Leaving directory `/home/hamid/src/acousticbrainz/src/essentia/build'
Build failed
 -> task in 'essentia_standard_onsetrate' failed with exit status 1 (run with -v to display more information)
 -> task in 'essentia_standard_beatsmarker' failed with exit status 1 (run with -v to display more information)
 -> task in 'essentia_standard_mfcc' failed with exit status 1 (run with -v to display more information)
 -> task in 'essentia_standard_rhythmtransform' failed with exit status 1 (run with -v to display more information)
 -> task in 'essentia_standard_fadedetection' failed with exit status 1 (run with -v to display more information)

I have installed ffmpeg 4.2.2 with the following configuration:

ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-nonfree
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

So I guess -lfdk-aac sould be found

$ ldconfig -p |grep fdk-aac
    libfdk-aac.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfdk-aac.so.1
    libfdk-aac.so (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libfdk-aac.so

Running ./waf -vvv results in the following errors:

Waf: Leaving directory `/home/hamid/src/acousticbrainz/src/essentia/build'
  File "./waf", line 168, in <module>
    Scripting.waf_entry_point(cwd, VERSION, wafdir)
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Scripting.py", line 119, in waf_entry_point
    run_commands()
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Scripting.py", line 182, in run_commands
    ctx=run_command(cmd_name)
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Scripting.py", line 173, in run_command
    ctx.execute()
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Scripting.py", line 375, in execute
    return execute_method(self)
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Build.py", line 93, in execute
    self.execute_build()
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Build.py", line 100, in execute_build
    self.compile()
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Tools/errcheck.py", line 140, in check_compile
    ret=self.orig_compile()
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Build.py", line 176, in compile
    raise Errors.BuildError(self.producer.error)
  File "/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Errors.py", line 26, in __init__
    WafError.__init__(self,self.format_error())

Build failed
 -> task in 'essentia_standard_beatsmarker' failed with exit status 1: 
    {task 139889353502800: cxxprogram standard_beatsmarker.cpp.1.o -> essentia_standard_beatsmarker}
['/usr/bin/g++', '-static', '-pthread', '-pthread', '-pthread', '-pthread', '-pthread', 'src/examples/standard_beatsmarker.cpp.1.o', '-o/home/hamid/src/acousticbrainz/src/essentia/build/src/examples/essentia_standard_beatsmarker', '-Wl,-Bstatic', '-Lsrc', '-lessentia', '-Wl,-Bstatic', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L\\[QtCore\\]', '-lgaia2', '-lfftw3f', '-lavformat', '-lm', '-lbz2', '-lz', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavutil', '-lm', '-lavresample', '-lsamplerate', '-ltag', '-lyaml', '-lgaia2', '-lQtCore', '-lyaml', '-ldl', '-lchromaprint', '-Wl,-Bsymbolic']
 -> task in 'essentia_standard_fadedetection' failed with exit status 1: 
    {task 139889353504080: cxxprogram standard_fadedetection.cpp.2.o -> essentia_standard_fadedetection}
['/usr/bin/g++', '-static', '-pthread', '-pthread', '-pthread', '-pthread', '-pthread', 'src/examples/standard_fadedetection.cpp.2.o', '-o/home/hamid/src/acousticbrainz/src/essentia/build/src/examples/essentia_standard_fadedetection', '-Wl,-Bstatic', '-Lsrc', '-lessentia', '-Wl,-Bstatic', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L\\[QtCore\\]', '-lgaia2', '-lfftw3f', '-lavformat', '-lm', '-lbz2', '-lz', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavutil', '-lm', '-lavresample', '-lsamplerate', '-ltag', '-lyaml', '-lgaia2', '-lQtCore', '-lyaml', '-ldl', '-lchromaprint', '-Wl,-Bsymbolic']
 -> task in 'essentia_standard_onsetrate' failed with exit status 1: 
    {task 139889353504592: cxxprogram standard_onsetrate.cpp.4.o -> essentia_standard_onsetrate}
['/usr/bin/g++', '-static', '-pthread', '-pthread', '-pthread', '-pthread', '-pthread', 'src/examples/standard_onsetrate.cpp.4.o', '-o/home/hamid/src/acousticbrainz/src/essentia/build/src/examples/essentia_standard_onsetrate', '-Wl,-Bstatic', '-Lsrc', '-lessentia', '-Wl,-Bstatic', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L\\[QtCore\\]', '-lgaia2', '-lfftw3f', '-lavformat', '-lm', '-lbz2', '-lz', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavutil', '-lm', '-lavresample', '-lsamplerate', '-ltag', '-lyaml', '-lgaia2', '-lQtCore', '-lyaml', '-ldl', '-lchromaprint', '-Wl,-Bsymbolic']
 -> task in 'essentia_standard_pitchdemo' failed with exit status 1: 
    {task 139889353504848: cxxprogram standard_pitchdemo.cpp.5.o -> essentia_standard_pitchdemo}
['/usr/bin/g++', '-static', '-pthread', '-pthread', '-pthread', '-pthread', '-pthread', 'src/examples/standard_pitchdemo.cpp.5.o', '-o/home/hamid/src/acousticbrainz/src/essentia/build/src/examples/essentia_standard_pitchdemo', '-Wl,-Bstatic', '-Lsrc', '-lessentia', '-Wl,-Bstatic', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L\\[QtCore\\]', '-lgaia2', '-lfftw3f', '-lavformat', '-lm', '-lbz2', '-lz', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavutil', '-lm', '-lavresample', '-lsamplerate', '-ltag', '-lyaml', '-lgaia2', '-lQtCore', '-lyaml', '-ldl', '-lchromaprint', '-Wl,-Bsymbolic']
 -> task in 'essentia_standard_mfcc' failed with exit status 1: 
    {task 139889353504336: cxxprogram standard_mfcc.cpp.3.o -> essentia_standard_mfcc}
['/usr/bin/g++', '-static', '-pthread', '-pthread', '-pthread', '-pthread', '-pthread', 'src/examples/standard_mfcc.cpp.3.o', '-o/home/hamid/src/acousticbrainz/src/essentia/build/src/examples/essentia_standard_mfcc', '-Wl,-Bstatic', '-Lsrc', '-lessentia', '-Wl,-Bstatic', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L\\[QtCore\\]', '-lgaia2', '-lfftw3f', '-lavformat', '-lm', '-lbz2', '-lz', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavutil', '-lm', '-lavresample', '-lsamplerate', '-ltag', '-lyaml', '-lgaia2', '-lQtCore', '-lyaml', '-ldl', '-lchromaprint', '-Wl,-Bsymbolic']
 -> task in 'essentia_standard_rhythmtransform' failed with exit status 1: 
    {task 139889353505104: cxxprogram standard_rhythmtransform.cpp.6.o -> essentia_standard_rhythmtransform}
['/usr/bin/g++', '-static', '-pthread', '-pthread', '-pthread', '-pthread', '-pthread', 'src/examples/standard_rhythmtransform.cpp.6.o', '-o/home/hamid/src/acousticbrainz/src/essentia/build/src/examples/essentia_standard_rhythmtransform', '-Wl,-Bstatic', '-Lsrc', '-lessentia', '-Wl,-Bstatic', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L\\[QtCore\\]', '-lgaia2', '-lfftw3f', '-lavformat', '-lm', '-lbz2', '-lz', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavutil', '-lm', '-lavresample', '-lsamplerate', '-ltag', '-lyaml', '-lgaia2', '-lQtCore', '-lyaml', '-ldl', '-lchromaprint', '-Wl,-Bsymbolic']
 -> task in 'essentia_standard_vibrato' failed with exit status 1: 
    {task 139889353505616: cxxprogram standard_vibrato.cpp.8.o -> essentia_standard_vibrato}
['/usr/bin/g++', '-static', '-pthread', '-pthread', '-pthread', '-pthread', '-pthread', 'src/examples/standard_vibrato.cpp.8.o', '-o/home/hamid/src/acousticbrainz/src/essentia/build/src/examples/essentia_standard_vibrato', '-Wl,-Bstatic', '-Lsrc', '-lessentia', '-Wl,-Bstatic', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L\\[QtCore\\]', '-lgaia2', '-lfftw3f', '-lavformat', '-lm', '-lbz2', '-lz', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavutil', '-lm', '-lavresample', '-lsamplerate', '-ltag', '-lyaml', '-lgaia2', '-lQtCore', '-lyaml', '-ldl', '-lchromaprint', '-Wl,-Bsymbolic']
 -> task in 'essentia_standard_spectralcontrast' failed with exit status 1: 
    {task 139889353505360: cxxprogram standard_spectralcontrast.cpp.7.o -> essentia_standard_spectralcontrast}
['/usr/bin/g++', '-static', '-pthread', '-pthread', '-pthread', '-pthread', '-pthread', 'src/examples/standard_spectralcontrast.cpp.7.o', '-o/home/hamid/src/acousticbrainz/src/essentia/build/src/examples/essentia_standard_spectralcontrast', '-Wl,-Bstatic', '-Lsrc', '-lessentia', '-Wl,-Bstatic', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L/usr/local/lib', '-L\\[QtCore\\]', '-lgaia2', '-lfftw3f', '-lavformat', '-lm', '-lbz2', '-lz', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavcodec', '-lm', '-llzma', '-lz', '-lfdk-aac', '-lmp3lame', '-lm', '-lopus', '-ltheoraenc', '-ltheoradec', '-logg', '-lvorbis', '-lvorbisenc', '-lx264', '-lswresample', '-lm', '-lavutil', '-lm', '-lavutil', '-lm', '-lavresample', '-lsamplerate', '-ltag', '-lyaml', '-lgaia2', '-lQtCore', '-lyaml', '-ldl', '-lchromaprint', '-Wl,-Bsymbolic']
  File './waf', line 168, in <module>
    Scripting.waf_entry_point(cwd, VERSION, wafdir)
  File '/home/hamid/src/acousticbrainz/src/essentia/.waf-2.0.18-77b2fa32f72970f15a949a4a564841e2/waflib/Scripting.py', line 131, in waf_entry_point
    Logs.error(e.msg)

Can you please help to figure it out?

alastair commented 4 years ago

You should try and work out what essentia configuration flags you need for your requirements:

./waf configure --with-vamp --with-gaia --with-examples --with-static-examples --with-example=streaming_extractor_music,streaming_extractor_freesound,streaming_extractor_music_svm

--with-examples --with-example= these two arguments are complimentary. --with-examples will compile all essentia examples. --with-example=x will compile just the specific ones that you want. If you only need the 3 extractors, only specify this argument

--with-static-examples this will compile the extractors as a static file, and therefore needs the dependencies to be compiled statically too. We provide tools for this in the packaging folder. If you don't specifically need statically compiled tools, remove this option

halaei commented 4 years ago

Thanks for the help. I have already find out about --with-examples and --with-example=. I didn't know about --with-static-examples and packaging folder. I will take a look at. I also removed --with-static-examples for now, and choose to have only one example, so my config looks like:

./waf configure --with-vamp --with-gaia --with-example=streaming_extractor_music,streaming_extractor_freesound,streaming_extractor_music_svm

However, I still have the following errors:

[300/300] Linking build/src/libessentia.so
/usr/bin/ld: /usr/local/lib/libavcodec.a(vc1dsp_mmx.o): relocation R_X86_64_PC32 against symbol `ff_pw_9' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

Waf: Leaving directory `/mnt/data/acousticbrainz/src/essentia/build'
Build failed
 -> task in 'essentia' failed with exit status 1 (run with -v to display more information)

I guess it means I have to recompile ffmpeg with --enable-pic.

halaei commented 4 years ago

I tested the packaging and followed the instruction in Linux/OSX static builds faq. Build was successful but still music extractor doesn't have gaia support:

[ WARNING  ] MusicExtractor: Gaia library is missing. Skipping configuration of SVM models.

Here is what I did:

packaging/build_3rdparty_static_debian.sh --with-gaia
./waf configure  --with-static-examples
./waf

Also I want to note that I tried ./waf configure --with-gaia --with-static-examples as well, which resulted link errors:

/usr/bin/ld: cannot find -lQtCore
collect2: error: ld returned 1 exit status

/usr/bin/ld: cannot find -lQtCore
collect2: error: ld returned 1 exit status

/usr/bin/ld: cannot find -lQtCore
collect2: error: ld returned 1 exit status

Waf: Leaving directory `/mnt/data/acousticbrainz/src/essentia/build'
Build failed
 -> task in 'essentia_standard_beatsmarker' failed with exit status 1 (run with -v to display more information)
 -> task in 'essentia_standard_mfcc' failed with exit status 1 (run with -v to display more information)
 -> task in 'essentia_standard_fadedetection' failed with exit status 1 (run with -v to display more information)

Any idea what should be done @alastair ?

halaei commented 4 years ago

I think packaging/build_3rdparty_static_debian.sh --with-gaia didn't build qt and gaia as said in faq. So I installed them manually. Probably a bug in faq page or build_3rdparty_static_debian.sh.

Now I can run ./essentia_streaming_extractor_music_svm with Segmentation fault :)