search

sfztools / sfizz

SFZ parser and synth c++ library, providing a JACK standalone client
https://sfz.tools/sfizz/
BSD 2-Clause "Simplified" License
394 stars 58 forks source link

How to build a static library for mingw64, Linux, and MacOS with no shared library dependencies? #448

Closed isabelgk closed 3 years ago

isabelgk commented 3 years ago

Hi there!

I'm working on building a VCV Rack module using sfizz as a dependency. It will be a GPL-v3 licensed module.

I have a few requirements for my project that lead me to my title question:

  1. no dependencies on shared libraries (static libraries are okay)
  2. compatible with the VCV Rack build environment

Is it possible to package up all of sfizz and libsndfile (since that's the main shared dependency of sfizz) into a static library? Once I have such a thing, incorporating it into my module would be possible.

jpcima commented 3 years ago

Hi @isabelgk It's no different of problems we deal with for shipping our own plugins, and can be a little difficult.

We have different strategies for different OS, hopefully it can be helpful. For a summary:

(1) https://git.io/JUrZK (2) attempting static link in minGW: -Wl,-Bstatic -lsfizz $(pkg-config --static --libs sndfile) -Wl,-Bdynamic

isabelgk commented 3 years ago

Thank you @jpcima ! This helps me get started.

A follow-up question... I am starting on my msys2 system. I have libsndfile installed using prebuilts from the msys2 package manager. Then I ran the following from /sfizz:

cmake -G "Unix Makefiles" \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_LTO=OFF \
-DSFIZZ_JACK=OFF \
-DSFIZZ_VST=OFF \
-DSFIZZ_STATIC_DEPENDENCIES=ON \
-DCMAKE_CXX_STANDARD=11 \
-DSFIZZ_SHARED=OFF \
-DSFIZZ_LV2=OFF \
-DSFIZZ_LV2_UI=OFF \
..
make

This build succeeds. :) From this, I have some .a files throughout the build directory. Do I need to gather all of them, or do I only need the libsfizz.a and libsfizz.hpp for my other project's Windows build? This may have an obvious answer, but I'm still getting the hang of libraries with C++.

jpcima commented 3 years ago

Admittedly, I forgot entirely of this problem of multiple .a files, considering we are building out plugins from within cmake, and of course Rack SDK cannot do that.

The answer's yes, it needs to link all the .a files.

In principle, we can come up with a clean solution based on Makefiles. At the project root, we can provide a file rack.mk that Rack projects can include to get static sfizz. It's about the same as we do already for dpf.mk.

I'll setup Rack and give this a try shortly.

jpcima commented 3 years ago

The latest has an experimental Makefile that includes sfizz in Rack builds. There is instructions how to use it in the file: https://github.com/sfztools/sfizz/blob/develop/rack.mk Give it a check.

isabelgk commented 3 years ago

Thanks, this is great!

Can you help me understand what I might need to customize for the variables in step 3?

Without custom variables, this is the error I find:

Package sndfile was not found in the pkg-config search path.
Perhaps you should add the directory containing `sndfile.pc'
to the PKG_CONFIG_PATH environment variable
No package 'sndfile' found
Package sndfile was not found in the pkg-config search path.
Perhaps you should add the directory containing `sndfile.pc'
to the PKG_CONFIG_PATH environment variable
No package 'sndfile' found
Package sndfile was not found in the pkg-config search path.
Perhaps you should add the directory containing `sndfile.pc'
to the PKG_CONFIG_PATH environment variable
No package 'sndfile' found
Package sndfile was not found in the pkg-config search path.
Perhaps you should add the directory containing `sndfile.pc'
to the PKG_CONFIG_PATH environment variable
No package 'sndfile' found
g++ -std=c++11 -Wsuggest-override  -Isrc -Ilib -Imodules -fPIC -ID:/dev/music/vcv/Rack-SDK-1.1.6/Rack-SDK/include -ID:/dev/music/vcv/Rack-SDK-1.1.6/Rack-SDK/dep/include -MMD -MP -g -march=nocona -ffast-math -fno-finite-math-only
-Wall -Wextra -Wno-unused-parameter -DARCH_WIN -D_USE_MATH_DEFINES  -I/d/dev/music/IggyLabsModules/dep/sfizz//src -I/d/dev/music/IggyLabsModules/dep/sfizz//src/sfizz -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external  -I/d/dev/music/IggyLabsModules/dep/sfizz//external/abseil-cpp -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/spline -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/cpuid/src -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/cpuid/platform/src -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/pugixml/src -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/kiss_fft -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/kiss_fft/tools
-I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/tunings/include -I/d/dev/music/IggyLabsModules/dep/sfizz//external/jsl/include -std=c++11 -Wsuggest-override  -Isrc -Ilib -Imodules -fPIC -ID:/dev/music/vcv/Rack-SDK-1.1.6/Rack-SDK/include -ID:/dev/music/vcv/Rack-SDK-1.1.6/Rack-SDK/dep/include -MMD -MP -g -march=nocona -ffast-math -fno-finite-math-only -Wall -Wextra -Wno-unused-parameter -DARCH_WIN -D_USE_MATH_DEFINES  -I/d/dev/music/IggyLabsModules/dep/sfizz//src -I/d/dev/music/IggyLabsModules/dep/sfizz//src/sfizz -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external  -I/d/dev/music/IggyLabsModules/dep/sfizz//external/abseil-cpp -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/spline -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/cpuid/src -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/cpuid/platform/src -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/pugixml/src -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/kiss_fft -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/kiss_fft/tools  -I/d/dev/music/IggyLabsModules/dep/sfizz//src/external/tunings/include -I/d/dev/music/IggyLabsModules/dep/sfizz//external/jsl/include -c -o /d/dev/music/IggyLabsModules/build/sfizz/src/sfizz/ADSREnvelope.cpp.o /d/dev/music/IggyLabsModules/dep/sfizz//src/sfizz/ADSREnvelope.cpp
In file included from D:/dev/music/IggyLabsModules/dep/sfizz/src/sfizz/CCMap.h:9,
                 from D:/dev/music/IggyLabsModules/dep/sfizz/src/sfizz/Region.h:8,
                 from D:/dev/music/IggyLabsModules/dep/sfizz/src/sfizz/ADSREnvelope.h:9,
                 from D:/dev/music/IggyLabsModules/dep/sfizz/src/sfizz/ADSREnvelope.cpp:7:
D:/dev/music/IggyLabsModules/dep/sfizz/src/sfizz/SfzHelpers.h:8:10: fatal error: absl/types/optional.h: No such file or directory
    8 | #include <absl/types/optional.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [dep/sfizz/rack.mk:88: /d/dev/music/IggyLabsModules/build/sfizz/src/sfizz/ADSREnvelope.cpp.o] Error 1

For reference, 

$ echo $PKG_CONFIG_PATH
/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig
$ echo $(pkg-config --static --libs sndfile)
-LC:/msys64/mingw64/lib -lsndfile -lFLAC -logg -lvorbis -lvorbisenc -lspeex
jpcima commented 3 years ago

fatal error: absl/types/optional.h: No such file or directory

For this one, it is most likely the sfizz folder is lacking the recursive submodules. I forgot mentioning this in the text. A call git submodule update --init --recursive should resolve this.

Can you help me understand what I might need to customize for the variables in step 3?

With these being optional, it's expected that not setting them will produce a working build, except not static. For a statically linked build, perhaps following overrides would do.

SFIZZ_PKG_CONFIG = pkg-config --static
SFIZZ_SNDFILE_LINK_FLAGS = -Wl,-Bstatic $(shell $(SFIZZ_PKG_CONFIG) --libs sndfile) -Wl,-Bdynamic
isabelgk commented 3 years ago

For this one, it is most likely the sfizz folder is lacking the recursive submodules. I forgot mentioning this in the text. A call git submodule update --init --recursive should resolve this.

Thanks, yes, this was what I needed there. I should have noticed that one. 😄

I think building is almost there. I think I just need to fix something with the DBG macro. Full output is here.

jpcima commented 3 years ago

Ok, it seems that's when filesystem::path are backed by wide-string, on Windows OS. The operation ostream<<wstring does not build. I pushed a hopeful fix for this case of DBG.

Then there's the equality comparison, at absl::EqualsIgnoreCase, also non-working with widestrings. I'll come up with a quick solution.

jpcima commented 3 years ago

There is a pair of fixes in branch develop. (hopefully, I can't test right away)

isabelgk commented 3 years ago

Thanks for the continued quick support on this!

Now I see:

D:/dev/music/IggyLabsModules/dep/sfizz/src/sfizz/FilePool.cpp:136:10: error: #if with no expression
  136 | #if WIN32
      |          ^
make: *** [dep/sfizz/rack.mk:88: build/sfizz/src/sfizz/FilePool.cpp.o] Error 1

If I change WIN32 to _WIN32 or false, I get:

g++ -o plugin.dll build/src/plugin.cpp.o build/src/model/more-ideas-model.cpp.o build/src/modules/fizz.cpp.o build/src/modules/more-ideas.cpp.o build/src/modules/select.cpp.o build/src/modules/table.cpp.o build/sfizz/libsfizz.a -shared -L/d/dev/music/vcv/Rack-SDK-1.1.6/Rack-SDK -lRack  -Wl,-Bstatic -LC:/msys64/mingw64/lib -lsndfile -lFLAC -logg -lvorbis -lvorbisenc -lspeex -Wl,-Bdynamic
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: build/sfizz/libsfizz.a(symbolize.cc.o): in function `absl::lts_2020_02_25::InitializeSymbolizer(char const*)':
D:/dev/music/IggyLabsModules/dep/sfizz/external/abseil-cpp/absl/debugging/symbolize_win32.inc:47: undefined reference to `__imp_SymSetOptions'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: D:/dev/music/IggyLabsModules/dep/sfizz/external/abseil-cpp/absl/debugging/symbolize_win32.inc:48: undefined reference to `__imp_SymInitialize'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: build/sfizz/libsfizz.a(symbolize.cc.o): in function `absl::lts_2020_02_25::Symbolize(void const*, char*, int)':
D:/dev/music/IggyLabsModules/dep/sfizz/external/abseil-cpp/absl/debugging/symbolize_win32.inc:65: undefined reference to `__imp_SymFromAddr'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/mingw64/lib\libvorbis.a(block.o):(.text+0x208): undefined reference to `oggpack_writeinit'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/mingw64/lib\libvorbis.a(block.o):(.text+0x223): undefined reference to `oggpack_writeinit'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/mingw64/lib\libvorbis.a(block.o):(.text+0x35a): undefined reference to `oggpack_writeclear'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/mingw64/lib\libvorbis.a(analysis.o):(.text+0x3b): undefined reference to `oggpack_reset'
...
jpcima commented 3 years ago

This has to be #if defined(_WIN32) for support of all compilers. I'll edit this one.

Regarding the link, we fixed an issue in libsndfile with the link order, in a static link situation. This is not in the current libsndfile release, it should land in the next. https://github.com/erikd/libsndfile/pull/515

A current workaround is that of manually editing sndfile.pc. This is a sndfile.pc which worked under our build environment.

prefix=/usr/i686-w64-mingw32
exec_prefix=${prefix}
libdir=/usr/i686-w64-mingw32/lib
includedir=/usr/i686-w64-mingw32/include

Name: sndfile
Description: A library for reading and writing audio files
Requires:
Requires.private: flac ogg vorbis vorbisenc
Version: 1.0.28
Libs: -L${libdir} -lsndfile
Cflags: -I${includedir}
isabelgk commented 3 years ago

Thanks! I ended up updating it to the following to include speex to avoid issues there.

prefix=/mingw64
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: sndfile
Description: A library for reading and writing audio files
Requires: 
Requires.private: flac ogg vorbis vorbisenc speex
Version: 1.0.28
Libs: -L${libdir} -lsndfile
Cflags: -I${includedir}

There looks like there are two small issues with Win32 and abseil I think. 

$ make
g++ -o plugin.dll build/src/plugin.cpp.o build/src/model/more-ideas-model.cpp.o build/src/modules/fizz.cpp.o build/src/modules/more-ideas.cpp.o build/src/modules/select.cpp.o build/src/modules/table.cpp.o build/sfizz/libsfizz.a -shared -L/d/dev/music/vcv/Rack-SDK-1.1.6/Rack-SDK -lRack  -Wl,-Bstatic -LC:/msys64/mingw64/lib -lsndfile -lFLAC -lm -lvorbisenc -lvorbis -lm -logg -lspeex -lm -Wl,-Bdynamic
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: build/sfizz/libsfizz.a(symbolize.cc.o): in function `absl::lts_2020_02_25::InitializeSymbolizer(char const*)':
D:/dev/music/IggyLabsModules/dep/sfizz/external/abseil-cpp/absl/debugging/symbolize_win32.inc:47: undefined reference to `__imp_SymSetOptions'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: D:/dev/music/IggyLabsModules/dep/sfizz/external/abseil-cpp/absl/debugging/symbolize_win32.inc:48: undefined reference to `__imp_SymInitialize'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: build/sfizz/libsfizz.a(symbolize.cc.o): in function `absl::lts_2020_02_25::Symbolize(void const*, char*, int)':
D:/dev/music/IggyLabsModules/dep/sfizz/external/abseil-cpp/absl/debugging/symbolize_win32.inc:65: undefined reference to `__imp_SymFromAddr'
collect2.exe: error: ld returned 1 exit status
jpcima commented 3 years ago

Try latest again, a link is added to dbghelp.dll on Win32. Hopefully none other is missing from now, but I don't have these build configurations to check on my side right now.

isabelgk commented 3 years ago

Build success! Thank you so very much!

I cannot check builds on Linux and Mac just now, but I may have a few questions on those in a couple of weeks. Again, I really appreciate the help and your work on this library.

paulfd commented 3 years ago

We've integrated a libsndfile replacement that can be statically linked in sfizz, so I suppose this would solve this one ! If you're on the develop branch you can check the SFIZZ_USE_SNDFILE configuration variable in cmake or the rack.mk file.

isabelgk commented 3 years ago

Thank you all! Apologies for radio silence for a while... been caught up with some other things in the meantime. This is super helpful, so thanks again!