Open humphd opened 2 years ago
Can you try git submodule init; git submodule update
?
That's it, thank you! Shall I send a PR to add a note to https://github.com/PowerDNS/pdns/blob/18581cc79e4d9830faa468bffff353f2e1152427/Docker-README.md#compose-example?
I'm onto a new problem with Lua:
❯ docker compose up --build
[+] Building 24.4s (33/89)
=> [pdns-recursor internal] load build definition from Dockerfile-recursor 0.0s
=> => transferring dockerfile: 3.90kB 0.0s
=> [pdns-auth internal] load build definition from Dockerfile-auth 0.0s
=> => transferring dockerfile: 4.39kB 0.0s
=> [pdns-dnsdist internal] load build definition from Dockerfile-dnsdist 0.0s
=> => transferring dockerfile: 3.91kB 0.0s
=> [pdns-recursor internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [pdns-dnsdist internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [pdns-auth internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [pdns-auth internal] load metadata for docker.io/library/debian:11-slim 24.2s
=> [pdns-recursor internal] load build context 0.1s
=> => transferring context: 265.06kB 0.1s
=> CACHED [pdns-dnsdist stage-1 1/11] FROM docker.io/library/debian:11-slim@sha256:e8ad0bc7d0ee6afd46e904780942033ab83b42b446b58efa88d 0.0s
=> CACHED [pdns-dnsdist builder 2/16] RUN apt-get update && apt-get -y dist-upgrade && apt-get clean 0.0s
=> CANCELED [pdns-recursor builder 3/16] RUN apt-get install -y --no-install-recommends devscripts equivs git curl && apt-get clean 23.8s
=> [pdns-auth internal] load build context 7.6s
=> => transferring context: 106.86MB 7.5s
=> [pdns-dnsdist internal] load build context 8.4s
=> => transferring context: 98.22MB 8.3s
=> [pdns-dnsdist stage-1 2/11] RUN apt-get update && apt-get -y dist-upgrade && apt-get clean 9.9s
=> CANCELED [pdns-dnsdist builder 3/16] RUN apt-get update && apt-get -y dist-upgrade && apt-get install -y --no-install-recommends 23.8s
=> CACHED [pdns-auth builder 3/19] RUN apt-get update && apt-get -y dist-upgrade && apt-get install -y --no-install-recommends devscr 0.0s
=> CACHED [pdns-auth builder 4/19] COPY builder-support /source/builder-support 0.0s
=> CACHED [pdns-auth builder 5/19] RUN mk-build-deps -i -t 'apt-get -y -o Debug::pkgProblemResolver=yes --no-install-recommends' /sour 0.0s
=> CACHED [pdns-auth builder 6/19] WORKDIR /source/ 0.0s
=> CACHED [pdns-auth builder 7/19] COPY pdns /source/pdns 0.0s
=> CACHED [pdns-auth builder 8/19] COPY modules /source/modules 0.0s
=> CACHED [pdns-auth builder 9/19] COPY codedocs /source/codedocs 0.0s
=> CACHED [pdns-auth builder 10/19] COPY docs /source/docs 0.0s
=> CACHED [pdns-auth builder 11/19] COPY build-aux /source/build-aux 0.0s
=> CACHED [pdns-auth builder 12/19] COPY m4 /source/m4 0.0s
=> CACHED [pdns-auth builder 13/19] COPY ext /source/ext 0.0s
=> [pdns-auth builder 14/19] COPY .git /source/.git 0.4s
=> [pdns-auth builder 15/19] ADD configure.ac Makefile.am /source/ 0.1s
=> [pdns-auth builder 16/19] COPY builder/helpers/set-configure-ac-version.sh /usr/local/bin 0.1s
=> [pdns-auth builder 17/19] RUN if [ "NO" = "YES" ]; then BUILDER_VERSION="$(IS_RELEASE=YES BUILDER_MODULES=authoritative ./bui 10.3s
=> [pdns-dnsdist stage-1 3/14] RUN apt-get install -y python3 python3-jinja2 python3-atomicwrites tini libcap2-bin ca-certificates && 13.7s
=> [pdns-auth stage-1 3/12] RUN apt-get install -y python3 python3-jinja2 sqlite3 tini libcap2-bin vim-tiny && apt-get clean 13.9s
=> [pdns-recursor stage-1 3/11] RUN apt-get install -y python3 python3-jinja2 tini libcap2-bin && apt-get clean 13.9s
=> ERROR [pdns-auth builder 18/19] RUN mkdir /build && LUAVER=$([ -z "${NO_LUA_JIT##*$(dpkg --print-architecture)*}" ] && echo 'lua 5.1s
------
> [pdns-auth builder 18/19] RUN mkdir /build && LUAVER=$([ -z "${NO_LUA_JIT##*$(dpkg --print-architecture)*}" ] && echo 'lua5.3' || echo 'luajit') && ./configure --with-lua=${LUAVER} --sysconfdir=/etc/powerdns --enable-option-checking=fatal --with-dynmodules='bind geoip gmysql godbc gpgsql gsqlite3 ldap lmdb lua2 pipe remote tinydns' --enable-tools --enable-ixfrdist --with-unixodbc-lib=/usr/lib/$(dpkg-architecture -q DEB_BUILD_GNU_TYPE) && make clean && make $MAKEFLAGS -C ext && make $MAKEFLAGS -C modules && make $MAKEFLAGS -C pdns && make -C pdns install DESTDIR=/build && make -C modules install DESTDIR=/build && make clean && strip /build/usr/local/bin/* /build/usr/local/sbin/* /build/usr/local/lib/pdns/*.so:
#0 0.618 checking for a BSD-compatible install... /usr/bin/install -c
#0 0.633 checking whether build environment is sane... yes
#0 0.647 checking for a thread-safe mkdir -p... /bin/mkdir -p
#0 0.653 checking for gawk... no
#0 0.653 checking for mawk... mawk
#0 0.653 checking whether make sets $(MAKE)... yes
#0 0.669 checking whether make supports nested variables... yes
#0 0.684 checking whether UID '0' is supported by ustar format... yes
#0 0.684 checking whether GID '0' is supported by ustar format... yes
#0 0.684 checking how to create a ustar tar archive... gnutar
#0 0.715 checking whether make supports nested variables... (cached) yes
#0 0.716 checking whether make supports the include directive... yes (GNU style)
#0 0.755 checking for gcc... gcc
#0 0.798 checking whether the C compiler works... yes
#0 0.846 checking for C compiler default output file name... a.out
#0 0.847 checking for suffix of executables...
#0 0.889 checking whether we are cross compiling... no
#0 1.002 checking for suffix of object files... o
#0 1.027 checking whether we are using the GNU C compiler... yes
#0 1.051 checking whether gcc accepts -g... yes
#0 1.074 checking for gcc option to accept ISO C89... none needed
#0 1.113 checking whether gcc understands -c and -o together... yes
#0 1.193 checking dependency style of gcc... gcc3
#0 1.242 checking how to run the C preprocessor... gcc -E
#0 1.308 checking for grep that handles long lines and -e... /bin/grep
#0 1.313 checking for egrep... /bin/grep -E
#0 1.315 checking for ANSI C header files... yes
#0 1.431 checking for sys/types.h... yes
#0 1.470 checking for sys/stat.h... yes
#0 1.512 checking for stdlib.h... yes
#0 1.563 checking for string.h... yes
#0 1.603 checking for memory.h... yes
#0 1.654 checking for strings.h... yes
#0 1.692 checking for inttypes.h... yes
#0 1.742 checking for stdint.h... yes
#0 1.784 checking for unistd.h... yes
#0 1.824 checking minix/config.h usability... no
#0 1.856 checking minix/config.h presence... no
#0 1.876 checking for minix/config.h... no
#0 1.876 checking whether it is safe to define __EXTENSIONS__... yes
#0 1.937 checking build system type... aarch64-unknown-linux-gnu
#0 2.005 checking host system type... aarch64-unknown-linux-gnu
#0 2.006 checking for gcc... (cached) gcc
#0 2.032 checking whether we are using the GNU C compiler... (cached) yes
#0 2.032 checking whether gcc accepts -g... (cached) yes
#0 2.033 checking for gcc option to accept ISO C89... (cached) none needed
#0 2.033 checking whether gcc understands -c and -o together... (cached) yes
#0 2.033 checking dependency style of gcc... (cached) gcc3
#0 2.034 checking for g++... g++
#0 2.064 checking whether we are using the GNU C++ compiler... yes
#0 2.108 checking whether g++ accepts -g... yes
#0 2.163 checking dependency style of g++... gcc3
#0 2.211 checking size of time_t... 8
#0 2.306 checking whether time_t is signed... yes
#0 2.353 checking for bison... bison -y
#0 2.353 checking if bison is the parser generator... yes
#0 2.357 checking for flex... flex
#0 2.368 checking lex output file root... lex.yy
#0 2.368 checking lex library... none needed
#0 2.530 checking whether yytext is a pointer... no
#0 2.581 checking if the lexer is flex... yes
#0 2.590 checking whether g++ supports C++17 features with -std=c++17... yes
#0 2.686 checking how to print strings... printf
#0 2.689 checking for a sed that does not truncate output... /bin/sed
#0 2.697 checking for fgrep... /bin/grep -F
#0 2.700 checking for ld used by gcc... /usr/bin/ld
#0 2.707 checking if the linker (/usr/bin/ld) is GNU ld... yes
#0 2.710 checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
#0 2.714 checking the name lister (/usr/bin/nm -B) interface... BSD nm
#0 2.746 checking whether ln -s works... yes
#0 2.746 checking the maximum length of command line arguments... 1572864
#0 2.775 checking how to convert aarch64-unknown-linux-gnu file names to aarch64-unknown-linux-gnu format... func_convert_file_noop
#0 2.775 checking how to convert aarch64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
#0 2.775 checking for /usr/bin/ld option to reload object files... -r
#0 2.775 checking for objdump... objdump
#0 2.775 checking how to recognize dependent libraries... pass_all
#0 2.777 checking for dlltool... no
#0 2.777 checking how to associate runtime and link libraries... printf %s\n
#0 2.777 checking for ar... ar
#0 2.778 checking for archiver @FILE support... @
#0 2.820 checking for strip... strip
#0 2.822 checking for ranlib... ranlib
#0 2.823 checking command to parse /usr/bin/nm -B output from gcc object... ok
#0 3.063 checking for sysroot... no
#0 3.066 checking for a working dd... /bin/dd
#0 3.081 checking how to truncate binary pipes... /bin/dd bs=4096 count=1
#0 3.091 checking for mt... no
#0 3.092 checking if : is a manifest tool... no
#0 3.103 checking for dlfcn.h... yes
#0 3.211 checking for objdir... .libs
#0 3.309 checking if gcc supports -fno-rtti -fno-exceptions... no
#0 3.381 checking for gcc option to produce PIC... -fPIC -DPIC
#0 3.381 checking if gcc PIC flag -fPIC -DPIC works... yes
#0 3.445 checking if gcc static flag -static works... yes
#0 3.519 checking if gcc supports -c -o file.o... yes
#0 3.554 checking if gcc supports -c -o file.o... (cached) yes
#0 3.554 checking whether the gcc linker (/usr/bin/ld) supports shared libraries... yes
#0 3.569 checking whether -lc should be explicitly linked in... no
#0 3.606 checking dynamic linker characteristics... GNU/Linux ld.so
#0 3.699 checking how to hardcode library paths into programs... immediate
#0 3.699 checking for shl_load... no
#0 3.752 checking for shl_load in -ldld... no
#0 3.809 checking for dlopen... no
#0 3.861 checking for dlopen in -ldl... yes
#0 3.901 checking whether a program can dlopen itself... yes
#0 3.953 checking whether a statically linked program can dlopen itself... no
#0 4.041 checking whether stripping libraries is possible... yes
#0 4.046 checking if libtool supports shared libraries... yes
#0 4.046 checking whether to build shared libraries... yes
#0 4.046 checking whether to build static libraries... no
#0 4.051 checking how to run the C++ preprocessor... g++ -std=c++17 -E
#0 4.230 checking for ld used by g++ -std=c++17... /usr/bin/ld
#0 4.234 checking if the linker (/usr/bin/ld) is GNU ld... yes
#0 4.245 checking whether the g++ -std=c++17 linker (/usr/bin/ld) supports shared libraries... yes
#0 4.366 checking for g++ -std=c++17 option to produce PIC... -fPIC -DPIC
#0 4.366 checking if g++ -std=c++17 PIC flag -fPIC -DPIC works... yes
#0 4.403 checking if g++ -std=c++17 static flag -static works... yes
#0 4.478 checking if g++ -std=c++17 supports -c -o file.o... yes
#0 4.531 checking if g++ -std=c++17 supports -c -o file.o... (cached) yes
#0 4.531 checking whether the g++ -std=c++17 linker (/usr/bin/ld) supports shared libraries... yes
#0 4.531 checking dynamic linker characteristics... (cached) GNU/Linux ld.so
#0 4.539 checking how to hardcode library paths into programs... immediate
#0 4.539 checking whether -latomic is needed for __atomic builtins... no
#0 4.666 checking for pthread_np.h... no
#0 4.702 checking for 2-arg pthread_setname_np... yes
#0 4.773 checking whether strerror_r is declared... yes
#0 4.822 checking for strerror_r... yes
#0 4.895 checking whether strerror_r returns char *... yes
#0 4.949 checking for pkg-config... /usr/bin/pkg-config
#0 4.949 checking pkg-config is at least version 0.9.0... yes
#0 4.953 checking for grep that handles long lines and -e... (cached) /bin/grep
#0 4.954 checking which Lua implementation to use... lua5.3
#0 4.958 checking for lua5.3 >= 5.1... no
#0 4.968 configure: error: Selected Lua (lua5.3) not found
------
failed to solve: executor failed running [/bin/sh -c mkdir /build && LUAVER=$([ -z "${NO_LUA_JIT##*$(dpkg --print-architecture)*}" ] && echo 'lua5.3' || echo 'luajit') && ./configure --with-lua=${LUAVER} --sysconfdir=/etc/powerdns --enable-option-checking=fatal --with-dynmodules='bind geoip gmysql godbc gpgsql gsqlite3 ldap lmdb lua2 pipe remote tinydns' --enable-tools --enable-ixfrdist --with-unixodbc-lib=/usr/lib/$(dpkg-architecture -q DEB_BUILD_GNU_TYPE) && make clean && make $MAKEFLAGS -C ext && make $MAKEFLAGS -C modules && make $MAKEFLAGS -C pdns && make -C pdns install DESTDIR=/build && make -C modules install DESTDIR=/build && make clean && strip /build/usr/local/bin/* /build/usr/local/sbin/* /build/usr/local/lib/pdns/*.so]: exit code: 1
Shall I send a PR
yes please, thanks!
#0 4.968 configure: error: Selected Lua (lua5.3) not found
As you might have spotted, the Dockerfile switches from luajit to lua5.3 for certain architectures, like the arm64 you are on. I don't think we tested that after a0d3acff25a92627186ee43bead110aef416f59a - perhaps you can see if 10-slim works (note, change it in two places). Also perhaps this is enough hint for you to figure out what's breaking. If not, post again, we'll have a look.
OK, submitted #12180, thanks for the fast reply with the answer!
For the Lua issue, switching both stages to use debian:10-slim
works for me with M1.
For the existing debian:11-slim
stages, adding liblua5.3-dev
in the builder
stage got me past configure and then it builds/runs:
diff --git a/Dockerfile-auth b/Dockerfile-auth
index bde213083..ebab9a58c 100644
--- a/Dockerfile-auth
+++ b/Dockerfile-auth
@@ -9,7 +9,7 @@ ENV NO_LUA_JIT="s390x arm64"
RUN apt-get update && apt-get -y dist-upgrade && apt-get clean
# devscripts gives us mk-build-deps (and a lot of other stuff)
-RUN apt-get update && apt-get -y dist-upgrade && apt-get install -y --no-install-recommends devscripts dpkg-dev equivs git python3-venv && apt-get clean
+RUN apt-get update && apt-get -y dist-upgrade && apt-get install -y --no-install-recommends devscripts dpkg-dev equivs git liblua5.3-dev python3-venv && apt-get clean
# import everything - this could be pdns.git OR an auth tarball!
COPY builder-support /source/builder-support
I don't really understand the build well enough to know if this is worth filing as a separate thing or not.
I don't really understand the build well enough to know if this is worth filing as a separate thing or not.
It would be, but given that's the only problem you have now, I updated the title of this one to reflect it.
@Habbie if simply adding liblua5.3-dev
like I've done locally is sufficient, I can do a PR for that too. If you want it done conditionally based on arch or something, I'll leave it for someone else with more knowledge of your build.
@Habbie if simply adding
liblua5.3-dev
like I've done locally is sufficient, I can do a PR for that too. If you want it done conditionally based on arch or something, I'll leave it for someone else with more knowledge of your build.
It should have come automatically out of the mk-build-deps
call earlier in the file (which then expects https://github.com/PowerDNS/pdns/blob/master/builder-support/debian/authoritative/debian-buster/control#L19 to do the right thing). The question is why it did not.
If we (and I'd rather not) fixed this in the Dockerfile, it indeed would have to be conditional, but I'd rather fix it between the control
file and mk-build-deps
somewhere.
This is what I thought you'd say. OK, I'll leave it to the experts! Thanks for investing in the Docker setup, it's great to find it already so far along. I'm excited to start using it.
Always glad to see happy people :-)
I just ran into this issue today. Commenting so that it's still active and a known still problem :)
https://gitlab.light.kow.is/kowis-projects/pdns-docker-build/-/jobs/1902
Looking at the control files, I think the problem is straightforward, but I'm not completely sure I've got it.
I think the authoritative control file should look like the ones for dnsdist or recursor:
Instead of:
libluajit-5.1-dev (>= 2.1.0~beta3+dfsg-5.3) [amd64 arm64] | libluajit-5.1-dev [amd64] | liblua5.3-dev,
It should be
libluajit-5.1-dev (>= 2.1.0~beta3+dfsg-5.3) [!arm64 !s390x],
libluajit-5.1-dev [!arm64 !s390x],
liblua5.3-dev [arm64 s390x,
I think?
OK, submitted #12180, thanks for the fast reply with the answer!
For the Lua issue, switching both stages to use
debian:10-slim
works for me with M1.For the existing
debian:11-slim
stages, addingliblua5.3-dev
in thebuilder
stage got me past configure and then it builds/runs:diff --git a/Dockerfile-auth b/Dockerfile-auth index bde213083..ebab9a58c 100644 --- a/Dockerfile-auth +++ b/Dockerfile-auth @@ -9,7 +9,7 @@ ENV NO_LUA_JIT="s390x arm64" RUN apt-get update && apt-get -y dist-upgrade && apt-get clean # devscripts gives us mk-build-deps (and a lot of other stuff) -RUN apt-get update && apt-get -y dist-upgrade && apt-get install -y --no-install-recommends devscripts dpkg-dev equivs git python3-venv && apt-get clean +RUN apt-get update && apt-get -y dist-upgrade && apt-get install -y --no-install-recommends devscripts dpkg-dev equivs git liblua5.3-dev python3-venv && apt-get clean # import everything - this could be pdns.git OR an auth tarball! COPY builder-support /source/builder-support
I don't really understand the build well enough to know if this is worth filing as a separate thing or not.
This is missing curl: RUN apt-get update && apt-get -y dist-upgrade && apt-get install -y --no-install-recommends curl devscripts dpkg-dev equivs git liblua5.3-dev python3-venv && apt-get clean
Failure to include that will break when trying to download rust using curl, i.e.:
[builder 13/18] RUN /source/install_rust.sh: 0.190 /source/install_rust.sh: 39: curl: not found 0.190 /source/install_rust.sh: Downloading rust-1.75.0-aarch64-unknown-linux-gnu.tar.gz
Dockerfile-recursor:32
30 | 31 | COPY builder-support/helpers/install_rust.sh /source/install_rust.sh 32 | >>> RUN /source/install_rust.sh 33 | 34 | # build and install (TODO: before we hit this line, rearrange /source structure if we are coming from a tarball)
ERROR: failed to solve: process "/bin/sh -c /source/install_rust.sh" did not complete successfully: exit code: 127 [fedora@fedora pdns]$ docker build -t pdns-recursor-arm64:latest -f Dockerfile-recursor . [+] Building 398.2s (33/33) FINISHED`
Short description
Running
docker-compose up
on my M1 Mac is failing. I can run it for one or all of the containers, and they all fail the same way: For example, withauth
:Environment
Steps to reproduce
docker-compose up
COPY builder/helpers/set-configure-ac-version.sh /usr/local/bin
Expected behaviour
Based on the docs, I'd assume this would build and bring up all 3 containers to test locally. I'm surprised I have to do more to make it work, but it seems like I do.
Actual behaviour
builder/helpers/set-configure-ac-version.sh
doesn't exist, so it fails when trying toCOPY
it into the images.I must need to do a step before I can use
docker-compose up
? Perhaps the Docker instructions need to include that.Thanks for any tips.