Waf build system produces very fat binary for iOS

[lamtev]

I'd like to use some DSP algorithms well-implemented in essentia in my iOS app.

When I run

./waf configure --cross-compile-ios --lightweight= --fft=ACCELERATE --build-static
./waf build
./waf install

WAF produces 127 MB libessentia.a static library. Seems so much for one static library inside iOS app.

OK, I don't need all the algorithms, so I specify required ones:

--include-algos=FrameCutter,Windowing,FFT,Spectrum,PitchYinFFT,\
    Magnitude,PeakDetection,CartesianToPolar,BPF,Resample,LowPass,IIR \ 

The output is 17 MB libessentia.a. Better than 127 MB, but still too large.

Let's look into compiled binary:

$ lipo -detailed_info libessentia.a 
Fat header in: libessentia.a
fat_magic 0xcafebabe
nfat_arch 3
architecture armv7
    cputype CPU_TYPE_ARM
    cpusubtype CPU_SUBTYPE_ARM_V7
    offset 68
    size 8727840
    align 2^2 (4)
architecture arm64
    cputype CPU_TYPE_ARM64
    cpusubtype CPU_SUBTYPE_ARM64_ALL
    offset 8727912
    size 8630792
    align 2^3 (8)
architecture x86_64
    cputype CPU_TYPE_X86_64
    cpusubtype CPU_SUBTYPE_X86_64_ALL
    offset 17358704
    size 9320
    align 2^3 (8)

Unfortunately, It is fat and contains unnecessary x86_64 architecture. But It is just 10 KB overhead.

libessentia.a built for macOS with the same included algorithms is 3 MB.

What am I doing wrong? Please help to reduce binary size on iOS!

[lamtev]

I had to deal with the essentia code in order to leave only the necessary source files for the specified algorithms. After removing all unnecessary code I compiled it using Xcode. The result is fantastic:

$ lipo -detailed_info essentia 
Fat header in: essentia
fat_magic 0xcafebabe
nfat_arch 2
architecture armv7
    cputype CPU_TYPE_ARM
    cpusubtype CPU_SUBTYPE_ARM_V7
    offset 16384
    size 640912
    align 2^14 (16384)
architecture arm64
    cputype CPU_TYPE_ARM64
    cpusubtype CPU_SUBTYPE_ARM64_ALL
    offset 671744
    size 634620
    align 2^14 (16384)

Just 1.3 MB for lib with debug information included!!!

[dbogdanov]

Perhaps you may want to have a look at the flags for cross-compilation in the wscript. Any further feedback is welcome.

[YangLuYang]

@lamtev How to reduce the binary size on iOS ?

[lamtev]

@YangLuYang As I mentioned in the comment, I've removed all the code from essentia and libsamplerate that is not needed in our use-case

[YangLuYang]

@lamtev Can you please provide some detail tutorial-like steps? I specify required alogs. ./waf configure --cross-compile-ios --lightweight= --fft=ACCELERATE --build-static --include-algos=FFT,FrameCutter,Windowing,Spectrum,MelBands,Magnitude,TriangularBands, The output is 17 MB libessentia.a I used lipo -thin split archive file and ar -t see the sub module, but there is no libsamplerate

__.SYMDEF
algorithm.cpp.1.o
configurable.cpp.1.o
connector.cpp.1.o
debugging.cpp.1.o
essentia.cpp.1.o
essentiautil.cpp.1.o
parameter.cpp.1.o
pool.cpp.1.o
range.cpp.1.o
network.cpp.1.o
networkparser.cpp.1.o
accumulatoralgorithm.cpp.1.o
devnull.cpp.1.o
poolstorage.cpp.1.o
ringbufferinput.cpp.1.o
ringbufferoutput.cpp.1.o
ringbuffervectoroutput.cpp.1.o
sinkbase.cpp.1.o
sourcebase.cpp.1.o
streamingalgorithm.cpp.1.o
streamingalgorithmcomposite.cpp.1.o
streamingalgorithmwrapper.cpp.1.o
stringutil.cpp.1.o
asciidag.cpp.1.o
asciidagparser.cpp.1.o
FreesoundLowlevelDescriptors.cpp.1.o
FreesoundRhythmDescriptors.cpp.1.o
FreesoundSfxDescriptors.cpp.1.o
FreesoundTonalDescriptors.cpp.1.o
MusicLowlevelDescriptors.cpp.1.o
MusicRhythmDescriptors.cpp.1.o
MusicTonalDescriptors.cpp.1.o
synth_utils.cpp.1.o
essentia_algorithms_reg.cpp.1.o
framecutter.cpp.1.o
spectrum.cpp.1.o
ffta.cpp.1.o
iffta.cpp.1.o
magnitude.cpp.1.o
triangularbands.cpp.1.o
windowing.cpp.1.o
melbands.cpp.1.o
splineutil.cpp.1.o
gamma.cpp.1.o
hyperg.cpp.1.o
iv.cpp.1.o
polevl.cpp.1.o

What should I do next?

[lamtev]

@YangLuYang I performed the following steps:

1. Determine which other algorithms specified algorithms use
2. Take only that files and all their dependencies
3. Copy them into the Xcode project directly OR into your own Pod if you manage project dependencies using cocoapods
4. Don't forget to copy essentia_algorithms_reg.cpp which is produced by ./waf for specified algorithms to your project

[rahul140490]

@lamtev, I have a similar issue related to monoloader and auidoloader. After running - ./waf configure --cross-compile-ios --lightweight=libav,libsamplerate --fft=ACCELERATE --build-static --include-algos=FFT,FrameCutter,Windowing,Spectrum,MelBands,Magnitude,TriangularBands,MonoLoader,MFCC,AudioLoader the essentia.a file that I get produces some linker errors like - Undefined symbol: _av_frame_unref in Xcode. Would you please help me to find out what I am doing wrong here or if I missed something? Please note that I am building an iOS app.