Docker build for Ubuntu 20.04 is failing

[Pnaghavi]

I have a program that I am trying to fuzz which uses a docker container made on Ubuntu 20.04 with specific dependency issues that prevent me from using ubuntu 18.04. Could you please look into providing the Dockerfile and Make file you used to create the ubuntu 20.04 version of Fuzzolic?

The content below is the Dockerfile I am using:

FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive

RUN sed -i -- 's/# deb-src/deb-src/g' /etc/apt/sources.list && cat /etc/apt/sources.list

# dependencies
RUN apt update -y && \
    apt-get build-dep -y qemu-user && \ 
    apt install -y \
    llvm-8 clang-8 nano \
    qemu-user git libglib2.0-dev libfdt-dev \
    libpixman-1-dev zlib1g-dev libcapstone-dev \
    strace cmake python3 libprotobuf10 \
    libibverbs-dev libjpeg62-dev \
    libpng16-16 libjbig-dev \
    build-essential libtool-bin python3-dev \
    automake flex bison libglib2.0-dev \
    libpixman-1-dev clang \
    python3-setuptools llvm wget \
    llvm-dev g++ g++-multilib python \
    python-pip lsb-release gcc-4.8 g++-4.8 \
    llvm-3.9 cmake libc6 libstdc++6 \
    linux-libc-dev gcc-multilib \
    apt-transport-https libtool \
        libtool-bin wget \
        automake autoconf \
        bison git gdb dumb-init valgrind ninja-build \
    time xxd python3-pip && \
    apt clean && \
    rm -rf /var/lib/apt/lists/*

RUN groupadd --gid 1008 ubuntu \
    && useradd --uid 1008 --gid ubuntu --shell /bin/bash --create-home ubuntu

USER ubuntu

RUN pip install --user virtualenv
RUN python3 -m pip install --user pytest

COPY --chown=1008:1008 . /home/ubuntu/fuzzolic

WORKDIR /home/ubuntu/fuzzolic

# Build QEMU tracer
RUN cd tracer && ./configure --prefix=`pwd`/../build --target-list=x86_64-linux-user && make -j `nproc` 

# Build custom Z3
RUN cd solver/fuzzy-sat/fuzzolic-z3 && mkdir build && cd build && cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=`pwd`/dist && make -j `nproc` && make install

# Set environment vars for Z3
ENV C_INCLUDE_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/include
ENV LIBRARY_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/lib
ENV LD_LIBRARY_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/lib
ENV BASH_ENV=/home/ubuntu/.bashrc
RUN echo "export C_INCLUDE_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/include" >> $BASH_ENV
RUN echo "export LIBRARY_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/lib" >> $BASH_ENV
RUN echo "export LD_LIBRARY_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/lib" >> $BASH_ENV

# Create fuzzy-sat-CLI folder
RUN cd solver/fuzzy-sat && \
    git rev-parse HEAD > /tmp/revision && \
    git checkout master && \
    git submodule update && \
    cd ../.. && \
    cp -r solver/fuzzy-sat solver/fuzzy-sat-cli && \
    rm solver/fuzzy-sat-cli/.git && \
    cd solver/fuzzy-sat && \
    git checkout `cat /tmp/revision` && \
    git submodule update

# Build fuzzy-sat-CLI
RUN cd solver/fuzzy-sat-cli && make -j `nproc`

# Build fuzzy-sat
RUN cd solver/fuzzy-sat && make -j `nproc`

# Build solver frontend
RUN cd solver && cmake . && make -j `nproc`

# Build AFL++
RUN cd utils && git clone https://github.com/AFLplusplus/AFLplusplus.git && \
    cd AFLplusplus && git checkout 2dac4e7 && \
    git apply ../afl-showmap.c.patch && \
    make -j `nproc` all && cd qemu_mode && ./build_qemu_support.sh
ENV AFL_PATH=/home/ubuntu/fuzzolic/utils/AFLplusplus
RUN echo "export AFL_PATH=/home/ubuntu/fuzzolic/utils/AFLplusplus" >> $BASH_ENV

# Build fuzzolic tests
RUN cd tests && make

CMD bash

The content below is the Makefile I am using:

build: # --force-rm 
    (cd ../../ && git submodule sync && git submodule update --init)
    (cd ../../solver/fuzzy-sat && git fetch && git submodule sync && git submodule update --init)
    docker build -t ercoppa/fuzzolic-runner-v1-20 -f ./Dockerfile ../../

push:
    docker push ercoppa/fuzzolic-runner-v1-20

patch-afl:
    cd AFLplusplus && git diff > ../../../utils/afl-showmap.c.patch

[ercoppa]

Hi,

Could you please look into providing the Dockerfile and Make file you used to create the ubuntu 20.04 version of Fuzzolic?

What is not working with your Dockerfile? Is there any issue?

I will try to build a new container based on Ubuntu 20.04 in a few days.

[Pnaghavi]

Thanks a lot for getting back to me. There are problems with installing the following dependencies:

python-pip lsb-release gcc-4.8 g++-4.8 \
llvm-3.9 cmake libc6 libstdc++6 \

In the first line, it says the following packages are not found: python-pip gcc-4.8 g++-4.8 In the second line, it says the following packages are not found: llvm-3.9 I tried a little to look for replacement packages for the ones it couldn't find for Ubuntu 20.04 but didn't really test the build with them. Please let me know when you have the Ubuntu 20.04 container built. I am Looking forward to using and hopefully ultimately citing your awesome work!!

[ercoppa]

Hi, I am pushing on docker hub the container ercoppa/fuzzolic-runner-v1:ubuntu2004 based on this Dockerfile. Let me know if this solve your problems.

[Pnaghavi]

I am sorry, but the docker file doesn't work still. I need to build off the docker file because once it builds, I need to add a bunch of stuff to the end for another project, so I can't go off the one pushed on docker hub. I have to get the docker file working. I changed the make file before running it:

build: # --force-rm 
    (cd ../../ && git submodule sync && git submodule update --init)
    (cd ../../solver/fuzzy-sat && git fetch && git submodule sync && git submodule update --init)
    docker build -t ercoppa/fuzzolic-runner-v1 -f ./Dockerfile.Ubuntu2004 ../../

push:
    docker push ercoppa/fuzzolic-runner-v1

patch-afl:
    cd AFLplusplus && git diff > ../../../utils/afl-showmap.c.patch

Once I call make, I get the docker build messages below. It looks like gcc 4.8 is not installed but it is necessary to do the final Fuzzolic build

~/fuzzolic/docker/fuzzolic-runner$ make
(cd ../../ && git submodule sync && git submodule update --init)
Synchronizing submodule url for 'solver/fuzzy-sat'
Synchronizing submodule url for 'solver/xxHash'
Synchronizing submodule url for 'tracer'
(cd ../../solver/fuzzy-sat && git fetch && git submodule sync && git submodule update --init)
Synchronizing submodule url for 'fuzzolic-z3'
Synchronizing submodule url for 'utility/xxhash'
docker build -t ercoppa/fuzzolic-runner-v1 -f ./Dockerfile.Ubuntu2004 ../../
[+] Building 179.6s (24/24) FINISHED
 => [internal] load build definition from Dockerfile.Ubuntu2004                                                                            0.0s
 => => transferring dockerfile: 49B                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                          0.1s
 => => transferring context: 2B                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                          178.9s
 => [internal] load build context                                                                                                          0.2s
 => => transferring context: 617.53kB                                                                                                      0.1s
 => [ 1/20] FROM docker.io/library/ubuntu:20.04@sha256:0e0402cd13f68137edb0266e1d2c682f217814420f2d43d300ed8f65479b14fb                    0.0s
 => CACHED [ 2/20] RUN sed -i -- 's/# deb-src/deb-src/g' /etc/apt/sources.list && cat /etc/apt/sources.list                                0.0s
 => CACHED [ 3/20] RUN apt update -y &&  DEBIAN_FRONTEND=noninteractive apt-get build-dep -y qemu-user &&  DEBIAN_FRONTEND=noninteractive  0.0s
 => CACHED [ 4/20] RUN groupadd --gid 1008 ubuntu     && useradd --uid 1008 --gid ubuntu --shell /bin/bash --create-home ubuntu            0.0s
 => CACHED [ 5/20] RUN pip install --user virtualenv                                                                                       0.0s
 => CACHED [ 6/20] RUN python3 -m pip install --user pytest                                                                                0.0s
 => CACHED [ 7/20] COPY --chown=1008:1008 . /home/ubuntu/fuzzolic                                                                          0.0s
 => CACHED [ 8/20] WORKDIR /home/ubuntu/fuzzolic                                                                                           0.0s
 => CACHED [ 9/20] RUN cd tracer && ./configure --prefix=`pwd`/../build --target-list=x86_64-linux-user && make -j `nproc`                 0.0s
 => CACHED [10/20] RUN cd solver/fuzzy-sat/fuzzolic-z3 && mkdir build && cd build && cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_  0.0s
 => CACHED [11/20] RUN echo "export C_INCLUDE_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/include" >> /home/ubuntu  0.0s
 => CACHED [12/20] RUN echo "export LIBRARY_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/lib" >> /home/ubuntu/.bash  0.0s
 => CACHED [13/20] RUN echo "export LD_LIBRARY_PATH=/home/ubuntu/fuzzolic/solver/fuzzy-sat/fuzzolic-z3/build/dist/lib" >> /home/ubuntu/.b  0.0s
 => CACHED [14/20] RUN cd solver/fuzzy-sat &&  git rev-parse HEAD > /tmp/revision &&  git checkout master &&  git submodule update &&  cd  0.0s
 => CACHED [15/20] RUN cd solver/fuzzy-sat-cli && make -j `nproc`                                                                          0.0s
 => CACHED [16/20] RUN cd solver/fuzzy-sat && make -j `nproc`                                                                              0.0s
 => CACHED [17/20] RUN cd solver && cmake . && make -j `nproc`                                                                             0.0s
 => CACHED [18/20] RUN cd utils && git clone https://github.com/AFLplusplus/AFLplusplus.git &&  cd AFLplusplus && git checkout 2dac4e7 &&  0.0s
 => CACHED [19/20] RUN echo "export AFL_PATH=/home/ubuntu/fuzzolic/utils/AFLplusplus" >> /home/ubuntu/.bashrc                              0.0s
 => ERROR [20/20] RUN cd tests && make                                                                                                     0.4s
------
 > [20/20] RUN cd tests && make:
#24 0.357 gcc-4.8 -c switch.c -O0
#24 0.358 make: gcc-4.8: Command not found
#24 0.358 make: *** [Makefile:16: switch.o] Error 127
------
executor failed running [/bin/sh -c cd tests && make]: exit code: 2
make: *** [Makefile:4: build] Error 1

[ercoppa]

Please pull the last commit, it should contain the fixes.

[Pnaghavi]

Perfect!!! It works now!!! Thank you!!