Can't create Dockerfile for ubuntu with aarm64 architecture

[turian]

Unfortunately, /root/DawDreamer/thirdparty/libfaust/download_libfaust.sh only works for i86. (arm64 is not supported yet: https://github.com/grame-cncm/faust/issues/951)

I am tearing my hair out trying to build a Dockerfile to build everything under arm64 Ubuntu.

Here is what I currently have:

# Using Ubuntu 22.04 as the base image
FROM ubuntu:22.04

LABEL maintainer="lastname@gmail.com" \
      version="0.1" \
      description=""

# Set the working directory
WORKDIR /root/

# Set the environment variables
ENV LANG=C.UTF-8 \
    TZ=Etc/UTC \
    DEBIAN_FRONTEND=noninteractive \
    PYTHONLIBPATH=/usr/lib/python3.8 \
    PYTHONINCLUDEPATH=/usr/include/python3.8

# Setting the timezone and installing essential packages
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone \
    && apt-get update \
    && apt-get install -y lsb-release software-properties-common wget python3-pip python3-dev git build-essential cmake g++ make nasm curl unzip libgl1-mesa-dev \
    && bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" \
    && apt-get install -y lv2-c++-tools libgtkmm-2.4-1v5 pkg-config lv2-dev libgtkmm-2.4-dev \
       libsndfile1 libx11-dev libxrandr-dev libxinerama-dev libxrender-dev libxcomposite-dev libxcursor-dev libfreetype6-dev libsndfile1-dev \
       libvorbis-dev libopus-dev libflac-dev libasound2-dev alsa-utils

# Download and extract Faust
RUN wget https://github.com/grame-cncm/faust/releases/download/2.69.3/faust-2.69.3.tar.gz \
    && tar zxvf faust-2.69.3.tar.gz

# Install additional dependencies
RUN apt-get install -y libpolly-17-dev

# Set up symlink for llvm-config if necessary
RUN ln -s /usr/bin/llvm-config-17 /usr/bin/llvm-config

# Log LLVM version and installations
RUN echo "LLVM Versions Installed:" > /root/build_logs.txt \
    && apt list --installed | grep llvm >> /root/build_logs.txt \
    && echo "\nllvm-config version:" >> /root/build_logs.txt \
    && llvm-config --version >> /root/build_logs.txt

# Build Faust and log the process
RUN echo "\nBuilding Faust..." >> /root/build_logs.txt \
    && mkdir faust-2.69.3/build/lib \
    && cd faust-2.69.3/build/ \
    && { cmake . -DINCLUDE_LLVM=ON -DINCLUDE_STATIC=ON && make && make -f Make.llvm.static && make install; } >> /root/build_logs.txt 2>&1

# Clone repositories and install Python packages
RUN echo "\nCloning repositories and installing Python packages..." >> /root/build_logs.txt \
    && git clone https://github.com/zynthian/moog.git \
    && pip3 install pedalboard \
    && git clone https://github.com/DBraun/DawDreamer.git \
    && cd DawDreamer \
    && git submodule init \
    && git submodule update

# Ensure the DawDreamer/Builds/LinuxMakefile directory exists and list its contents for verification
RUN echo "\nChecking DawDreamer/Builds/LinuxMakefile directory..." >> /root/build_logs.txt \
    && ls -la DawDreamer/Builds/LinuxMakefile >> /root/build_logs.txt 2>&1

## Copy the build logs to the output directory
RUN cp /root/build_logs.txt /output/

## Run make with VERBOSE and CONFIG in the DawDreamer build directory and log the output
RUN echo "\nBuilding DawDreamer..." >> /root/build_logs.txt \
    && cd DawDreamer/Builds/LinuxMakefile \
    && make VERBOSE=1 CONFIG=Release LIBS="-lstdc++fs" CXXFLAGS="-I../../alsa-lib/include -I/usr/include/python3.10 -I$PYTHONINCLUDEPATH"  LDFLAGS="-L/__w/DawDreamer/DawDreamer/alsa-lib/src -L$PYTHONLIBPATH -L/root/faust-2.69.3/lib -L/root/faust-2.69.3/build/lib/" >> /root/build_logs.txt 2>&1

# Move the built library
RUN cd DawDreamer/Builds/LinuxMakefile && mv build/libdawdreamer.so ../../dawdreamer/dawdreamer.so

## Build and install the Python package and log the output
RUN cd DawDreamer && python3 setup.py build >> /root/build_logs.txt 2>&1
RUN cd DawDreamer && python3 setup.py install >> /root/build_logs.txt 2>&1

## Test import of DawDreamer and log the output
RUN { python3 -c "import dawdreamer"; } >> /root/build_logs.txt 2>&1 || echo "Import failed" >> /root/build_logs.txt

## Final command
#CMD ["cat", "/root/build_logs.txt"]

It ultimately fails with

 > [14/14] RUN python3 -c "import dawdreamer; print('Successfully imported dawdreamer')":
0.264 Traceback (most recent call last):
0.264   File "<string>", line 1, in <module>
0.264   File "/usr/local/lib/python3.8/dist-packages/dawdreamer-0.8.0-py3.8-linux-aarch64.egg/dawdreamer/__init__.py", line 4, in <module>
0.264     from .dawdreamer import *
0.264 ImportError: /usr/local/lib/python3.8/dist-packages/dawdreamer-0.8.0-py3.8-linux-aarch64.egg/dawdreamer/dawdreamer.so: undefined symbol: _Z16deleteDSPFactoryP16llvm_dsp_factory

I have been tearing my hair out trying to figure out why the linked libfaustwithllvm has a bogus symbol.

Any idea how to fix this?

[turian]

I also try the standard Dockerfile for DawDreamer, even though it for i86, hoping to run through emulation mode.

But now I get this:

import dawdreamer Traceback (most recent call last): File "", line 1, in File "/DawDreamer/dawdreamer/init.py", line 4, in from .dawdreamer import * ImportError: /DawDreamer/dawdreamer/dawdreamer.so: undefined symbol: PyThread_tss_create

[root@ad2496ee1c86 DawDreamer]# python Python 2.7.5 (default, Nov 14 2023, 16:14:06) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux2 Type "help", "copyright", "credits" or "license" for more information. import dawdreamer Traceback (most recent call last): File "", line 1, in File "dawdreamer/init.py", line 4, in from .dawdreamer import * ImportError: dawdreamer/dawdreamer.so: undefined symbol: PyInstanceMethod_Type

[turian]

Perhaps I can disable Faust in DawDreamer? I am only interested in loading presets and running them

[DBraun]

I haven't tried it in a while, but you should be able to skip everything Faust related by removing the BUILD_DAWDREAMER_FAUST preprocessor in DawDreamer.jucer and regenerating the Makefiles. Editing the makefile by other means should be doable do. I think I'll try to do more architecture support toward the end of December.

[DBraun]

In your first post I noticed that the script uses both python3.10 and python 3.8. Your second post uses Python 2.7.

If that doesn't fix it, something could be going wrong with make -f Make.llvm.static, so running that in isolation and looking at debug print statements or logs could help.

[turian]

Sorry my mistake, I have been experimenting with ubuntu22.04 and 20.04 and python2 and python3 and experiencing no success.

But I have the same error that I originally posted, sorry for not rigorously testing what I commented. I have tried all variation though.

I can remove the BUILD_DAWDREAMER_FAUST in DawDreamer.jucer but I don't know how to regenerate the Makefiles.

[DBraun]

No worries. Can you try removing "-DBUILD_DAWDREAMER_FAUST" in https://github.com/DBraun/DawDreamer/blob/main/Builds/LinuxMakefile/Makefile

[turian]

I tried that but it still has a lot of faust stuff it's pulling in:

8746e3c89c10 ~/DawDreamer/Builds/LinuxMakefile # make
Linking DawDreamer - Dynamic Library
/usr/bin/aarch64-unknown-linux-gnu-ld.bfd: cannot find -lfaustwithllvm: No such file or directory
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:122: build/libdawdreamer.so] Error 1

I've even resorted to trying to make a Gentoo Dockerfile. But Gentoo doesn't want to build static LLVM: https://discourse.llvm.org/t/llvm-static-libs/38255 https://lists.llvm.org/pipermail/llvm-dev/2015-September/090589.html

I would really love a way to build DawDreamer that doesn't depend upon faust, this has been really gnarly

[DBraun]

Sorry I overlooked another thing. Try removing -lfaustwithllvm from the Makefile too. If there are other issues just keep deleting anything Faust related from the Makefile.

[turian]

Okay so your suggestion helped. (p.s. building libfaustwithllvm is super gnarly and I couldn't figure it out after trying a variety of methods)

This Dockerfile builds and I can import dawdreamer:

# Using Ubuntu 22.04 as the base image
FROM ubuntu:22.04

LABEL maintainer="lastname@gmail.com" \
      version="0.1" \
      description=""

# Set the working directory
WORKDIR /root/

# Set the environment variables
ENV LANG=C.UTF-8 \
    TZ=Etc/UTC \
    DEBIAN_FRONTEND=noninteractive \
    PYTHONLIBPATH=/usr/lib/python3.10 \
    PYTHONINCLUDEPATH=/usr/include/python3.10

# Setting the timezone and installing essential packages
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone \
    && apt-get update \
    && apt-get install -y lsb-release software-properties-common wget python3-pip python3-dev git build-essential cmake g++ make nasm curl unzip libgl1-mesa-dev \
    && apt-get install -y lv2-c++-tools libgtkmm-2.4-1v5 pkg-config lv2-dev libgtkmm-2.4-dev \
       libsndfile1 libx11-dev libxrandr-dev libxinerama-dev libxrender-dev libxcomposite-dev libxcursor-dev libfreetype6-dev libsndfile1-dev \
       libvorbis-dev libopus-dev libflac-dev libasound2-dev alsa-utils
#    && bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" \

# Clone repositories and install Python packages
RUN git clone https://github.com/zynthian/moog.git \
    && pip3 install pedalboard \
    && git clone https://github.com/DBraun/DawDreamer.git \
    && cd DawDreamer \
    && git submodule init \
    && git submodule update

# Modify Makefile
RUN perl -i -pe 's/ -lfaustwithllvm//' DawDreamer/Builds/LinuxMakefile/Makefile
RUN perl -i -pe 's/ "-DBUILD_DAWDREAMER_FAUST"//' DawDreamer/Builds/LinuxMakefile/Makefile

# Build libsamplerate
RUN cd DawDreamer/thirdparty/libsamplerate && \
    cmake -DCMAKE_BUILD_TYPE=Release -Bbuild_release -DCMAKE_POSITION_INDEPENDENT_CODE=ON && \
    cd build_release && \
    make CONFIG=Release

# Build DawDreamer
RUN cd DawDreamer/Builds/LinuxMakefile \
    && make -j8 VERBOSE=1 CONFIG=Release LIBS="-lstdc++fs" CXXFLAGS="-I../../alsa-lib/include -I/usr/include/python3.10 -I$PYTHONINCLUDEPATH"  LDFLAGS="-L/__w/DawDreamer/DawDreamer/alsa-lib/src -L$PYTHONLIBPATH -L/root/faust-2.69.3/lib -L/root/faust-2.69.3/build/lib/"

# Move the built library
RUN cd DawDreamer/Builds/LinuxMakefile && mv build/libdawdreamer.so ../../dawdreamer/dawdreamer.so

RUN cd moog && CXXFLAGS="-I/usr/include/lv2-c++-tools" CFLAGS="-I/usr/include/lv2-c++-tools" make && make install

# Build and install the Python package
RUN cd DawDreamer && python3 setup.py build && python3 setup.py install

# Test import of DawDreamer
RUN { python3 -c "import dawdreamer"; }