FlagBrew / PKSM

Gen I to GenVIII save manager.
GNU General Public License v3.0
1.73k stars 174 forks source link

(build) fatal error: bzlib.h: No such file or directory #1049

Closed route1rodent closed 5 years ago

route1rodent commented 5 years ago

Describe the bug I get an error bzlib.h: No such file or directory when trying to build the master source code.

I was trying to build the CIA for Citra Emu, following the README instructions of this project (it also requires devkitPro's tex3ds btw) but I cannot get it to work properly.

For building this I am using a Docker image, to not pollute my computer. As you can see in the Dockerfile, the OS has all required packages and libraries (in theory). Am I missing something or is this a bug?

Information about your environment

How to reproduce Place the Dockerfile (from above) in the PKSM project dir and then run:

docker build -t pksm:latest .
docker run -ti --rm pksm:latest make all

This requires docker in your computer.

Additional context

Dockerfile

FROM ubuntu:18.04 AS pksm_dev

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    software-properties-common locales locales-all

ENV DEBIAN_FRONTEND=noninteractive

ENV LC_ALL en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US.UTF-8
ENV PYTHONIOENCODING=utf-8   # required for python3 utf-8 encoding

ENV DEVKITPRO=/opt/devkitpro
ENV DEVKITARM=${DEVKITPRO}/devkitARM
ENV DEVKITPPC=${DEVKITPRO}/devkitPPC
ENV PATH=${DEVKITPRO}/tools/bin:$PATH

RUN apt-get install -y --no-install-recommends \
    sudo git vim nano curl wget pkg-config make dpkg ca-certificates \
    software-properties-common pacman gpg gnupg2 libxml2 \
    python3 python3-pip tar unzip \
    nodejs npm pandoc \
    fontconfig libfreetype6 libfreetype6-dev libjpeg-turbo8 libxext6 \
    libxrender1 xfonts-75dpi xfonts-base libc6 zlib1g \
    checkinstall imagemagick libmagick++-dev \
    build-essential g++ libbz2-dev \
    && apt-get clean -y && rm -rf /var/lib/apt/lists/*

WORKDIR /downloads

# devkitPro
RUN echo 'cacert=/etc/ssl/certs/ca-certificates.crt' > ~/.curlrc && \
    curl -OL https://github.com/devkitPro/pacman/releases/download/devkitpro-pacman-1.0.1/devkitpro-pacman.deb \
    && sudo dpkg -i devkitpro-pacman.deb

# python dependencies
RUN pip3 install gitpython

RUN pkg-config --cflags freetype2

# devkitpro packages
RUN sudo dkp-pacman -S --noconfirm libctru citro3d citro2d 3ds-curl 3ds-pkg-config 3ds-sdl_mixer tex3ds

# 3dstool
RUN curl -OL https://github.com/dnasdw/3dstool/releases/download/v1.2.6/3dstool_linux_x86_64.tar.gz \
    && tar -zxvf 3dstool_linux_x86_64.tar.gz \
    && mv 3dstool /bin/3dstool \
    && chmod +x /bin/3dstool \
    && which 3dstool \
    && rm -rf /downloads/*

# bannertool
RUN curl -OL https://github.com/Steveice10/bannertool/releases/download/1.2.0/bannertool.zip \
    && unzip bannertool.zip \
    && mv linux-x86_64/bannertool /bin/bannertool \
    && chmod +x /bin/bannertool \
    && which bannertool \
    && rm -rf /downloads/*

# ctrtool and makerom (for CIA builds)
RUN curl -OL https://github.com/jakcron/Project_CTR/releases/download/v0.16/makerom_016_ctrtool.zip \
    && unzip makerom_016_ctrtool.zip \
    && mv Ubuntu/ctrtool /bin/ctrtool \
    && mv Ubuntu/makerom /bin/makerom \
    && chmod +x /bin/ctrtool \
    && chmod +x /bin/makerom \
    && which ctrtool \
    && which makerom \
    && rm -rf /downloads/*

# libpng12-0 (for wkhtmltox)
RUN curl -OL http://security.ubuntu.com/ubuntu/pool/main/libp/libpng/libpng12-0_1.2.54-1ubuntu1.1_amd64.deb \
    && sudo dpkg -i libpng12-0_1.2.54-1ubuntu1.1_amd64.deb \
    && rm -rf /downloads/*

# wkhtmltox
RUN curl -OL https://downloads.wkhtmltopdf.org/0.12/0.12.5/wkhtmltox_0.12.5-1.xenial_amd64.deb \
    && sudo dpkg -i wkhtmltox_0.12.5-1.xenial_amd64.deb \
    && rm -rf /downloads/*

# github-wikito-converter and httpserver
RUN npm install -g github-wikito-converter httpserver

RUN apt-get update && apt-get install -y --no-install-recommends \
    bzip2 libbz2-1.0 libbz2-dev

COPY . /home/pksm
WORKDIR /home/pksm

RUN git submodule update --init --recursive && \
    ls -la /home/pksm

Debug info

make -C 3ds CITRA_DEBUG=1 VERSION_MAJOR=7 VERSION_MINOR=0 VERSION_MICRO=1
make[1]: Entering directory '/home/pksm/3ds'
Cloning EventsGallery...
Creating data...
Generating scripts...
pksm.ttf
Generated font with 39 glyphs
ui_sheet.t3s
CloudAccess.cpp
`which ccache` arm-none-eabi-g++ -MMD -MP -MF /home/pksm/3ds/build/CloudAccess.d -g -Wall -Wextra -Wno-psabi -O3 -mword-relocations -fomit-frame-pointer -ffunction-sections -Wno-unused-parameter -march=armv6k -mtune=mpcore -mfloat-abi=hard -mtp=soft -DQUIRC_MAX_REGIONS=10000 -DUNIX_HOST -DUNIQUE_ID=0xEC100 -DCITRA_DEBUG=1 `arm-none-eabi-pkg-config SDL_mixer --cflags` `curl-config --cflags` -I/home/pksm/3ds/../common/include -I/home/pksm/3ds/../common/include/gui -I/home/pksm/3ds/../common/include/io -I/home/pksm/3ds/../common/include/utils -I/home/pksm/3ds/../core/include -I/home/pksm/3ds/../core/include/i18n -I/home/pksm/3ds/../core/include/personal -I/home/pksm/3ds/../core/include/pkx -I/home/pksm/3ds/../core/include/sav -I/home/pksm/3ds/../core/include/wcx -I/home/pksm/3ds/include -I/home/pksm/3ds/include/gui -I/home/pksm/3ds/include/gui/overlay -I/home/pksm/3ds/include/gui/screen -I/home/pksm/3ds/include/gui/scripts -I/home/pksm/3ds/include/io -I/home/pksm/3ds/include/titles -I/home/pksm/3ds/include/utils -I/home/pksm/3ds/../common/include/picoc -I/home/pksm/3ds/../common/include/quirc -I/home/pksm/3ds/../core/memecrypto -I/opt/devkitpro/libctru/include -I/opt/devkitpro/portlibs/3ds/include -I/home/pksm/3ds/build -DARM11 -D_3DS -D_GNU_SOURCE=1 -fno-rtti -fno-exceptions -std=gnu++17 -c /home/pksm/3ds/../common/source/CloudAccess.cpp -o CloudAccess.o
mysterygift.cpp
`which ccache` arm-none-eabi-g++ -MMD -MP -MF /home/pksm/3ds/build/mysterygift.d -g -Wall -Wextra -Wno-psabi -O3 -mword-relocations -fomit-frame-pointer -ffunction-sections -Wno-unused-parameter -march=armv6k -mtune=mpcore -mfloat-abi=hard -mtp=soft -DQUIRC_MAX_REGIONS=10000 -DUNIX_HOST -DUNIQUE_ID=0xEC100 -DCITRA_DEBUG=1 `arm-none-eabi-pkg-config SDL_mixer --cflags` `curl-config --cflags` -I/home/pksm/3ds/../common/include -I/home/pksm/3ds/../common/include/gui -I/home/pksm/3ds/../common/include/io -I/home/pksm/3ds/../common/include/utils -I/home/pksm/3ds/../core/include -I/home/pksm/3ds/../core/include/i18n -I/home/pksm/3ds/../core/include/personal -I/home/pksm/3ds/../core/include/pkx -I/home/pksm/3ds/../core/include/sav -I/home/pksm/3ds/../core/include/wcx -I/home/pksm/3ds/include -I/home/pksm/3ds/include/gui -I/home/pksm/3ds/include/gui/overlay -I/home/pksm/3ds/include/gui/screen -I/home/pksm/3ds/include/gui/scripts -I/home/pksm/3ds/include/io -I/home/pksm/3ds/include/titles -I/home/pksm/3ds/include/utils -I/home/pksm/3ds/../common/include/picoc -I/home/pksm/3ds/../common/include/quirc -I/home/pksm/3ds/../core/memecrypto -I/opt/devkitpro/libctru/include -I/opt/devkitpro/portlibs/3ds/include -I/home/pksm/3ds/build -DARM11 -D_3DS -D_GNU_SOURCE=1 -fno-rtti -fno-exceptions -std=gnu++17 -c /home/pksm/3ds/../common/source/mysterygift.cpp -o mysterygift.o
In file included from /home/pksm/3ds/../common/source/mysterygift.cpp:27:
/home/pksm/3ds/../common/include/mysterygift.hpp:38:10: fatal error: bzlib.h: No such file or directory
   38 | #include <bzlib.h>
      |          ^~~~~~~~~
compilation terminated.
/opt/devkitpro/devkitARM/base_rules:79: recipe for target 'mysterygift.o' failed
make[2]: *** [mysterygift.o] Error 1
Makefile:252: recipe for target 'no-deps' failed
make[1]: *** [no-deps] Error 2
make[1]: Leaving directory '/home/pksm/3ds'
Makefile:22: recipe for target '3ds' failed
make: *** [3ds] Error 2
svfeplvce commented 5 years ago

You are missing the 3ds-bzip2 package, I believe. Just download it via pacman and you should be a-okay :)

route1rodent commented 5 years ago

@xunknxwnx thanks, but 3dstools was also missing

@piepie62 please do not close issues unless they are confirmed fixed. This is clearly consequence of a documentation issue.

In the readme you mention the devkit dependencies but some are missing or have different names:

missing in the readme: 3ds-bzip2, 3dstools, tex3ds

full list (with actual names): libctru citro3d citro2d 3ds-curl 3ds-pkg-config 3ds-sdl_mixer 3ds-bzip2 3dstools tex3ds

piepie62 commented 5 years ago

I closed it because I added 3ds-bzip2. I will add the other two, though. As for the naming issue, we're not trying to provide the actual package names, but what they're actually documented as

piepie62 commented 5 years ago

Also, there is a devkitARM docker image provided by devkitPro; it's suggested that you use that instead of building your own

route1rodent commented 5 years ago

thanks, I didn't know

route1rodent commented 5 years ago

anyway this is the final Dockerfile that worked for me, in case it is useful for someone else:


FROM ubuntu:19.04 AS pksm_dev

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    build-essential software-properties-common locales locales-all \
    sudo vim git curl wget dpkg pkg-config ca-certificates make \
    pacman gpg gnupg2 libxml2 python3 python3-pip \
    tar unzip bzip2 xz-utils \
    bsdtar doxygen g++ glibc-source \
    && pip3 install gitpython \
    && apt-get clean -y && rm -rf /var/lib/apt/lists/*

ENV LC_ALL en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US.UTF-8
ENV PYTHONIOENCODING=utf-8

ENV DEVKITPRO=/opt/devkitpro
ENV DEVKITARM=${DEVKITPRO}/devkitARM
ENV DEVKITPPC=${DEVKITPRO}/devkitPPC
ENV PATH=${DEVKITPRO}/tools/bin:$PATH

ENV CITRA_DEBUG=1

WORKDIR /downloads

# download requirements
RUN echo 'cacert=/etc/ssl/certs/ca-certificates.crt' > ~/.curlrc \
    # devkitpro
    && mkdir /downloads/devkitpro && cd /downloads/devkitpro \
    && curl -OL https://github.com/devkitPro/pacman/releases/download/devkitpro-pacman-1.0.1/devkitpro-pacman.deb \
    && sudo dpkg -i devkitpro-pacman.deb \
    && dkp-pacman -Scc --noconfirm \
    # 3dstool
    && mkdir /downloads/3dstool && cd /downloads/3dstool \
    && curl -OL https://github.com/dnasdw/3dstool/releases/download/v1.2.6/3dstool_linux_x86_64.tar.gz \
    && tar -zxvf 3dstool_linux_x86_64.tar.gz \
    && mv 3dstool /bin/3dstool && chmod +x /bin/3dstool \
    # bannertool
    && mkdir /downloads/bannertool && cd /downloads/bannertool \
    && curl -OL https://github.com/Steveice10/bannertool/releases/download/1.2.0/bannertool.zip \
    && unzip bannertool.zip \
    && mv linux-x86_64/bannertool /bin/bannertool && chmod +x /bin/bannertool \
    # makerom and ctrtool (for CIA builds)
    && mkdir /downloads/makerom_ctrtool && cd /downloads/makerom_ctrtool \
    && curl -OL https://github.com/jakcron/Project_CTR/releases/download/v0.16/makerom_016_ctrtool.zip \
    && unzip makerom_016_ctrtool.zip \
    && mv Ubuntu/ctrtool /bin/ctrtool && chmod +x /bin/ctrtool \
    && mv Ubuntu/makerom /bin/makerom && chmod +x /bin/makerom \
    #
    && rm -rf /downloads/* \
    && echo 'downloads finished.'

# install devkitpro packages
RUN dkp-pacman -Syyu --noconfirm 3ds-dev \
    && sudo dkp-pacman -S --noconfirm libctru citro3d citro2d \
       3ds-curl 3ds-pkg-config 3ds-sdl_mixer 3ds-bzip2 tex3ds 3dstools 3dslink \
       general-tools \
    && dkp-pacman -Scc --noconfirm

COPY . /home/pksm
WORKDIR /home/pksm

# init git submodules
RUN git submodule update --init --recursive && ls -la /home/pksm

# build cia (under /home/pksm/3ds/out)
RUN make 3ds

Later I extract the built files using docker cp:

docker build -t pksm:latest . || exit 1

rm -rf ./build
mkdir -p ./build

docker rm -f pksm-build
docker create -t --name pksm-build pksm:latest /bin/bash
docker cp pksm-build:/home/pksm/3ds/out/ ./build/