search

PowerDNS / pdns

PowerDNS Authoritative, PowerDNS Recursor, dnsdist
https://www.powerdns.com/
GNU General Public License v2.0
3.49k stars 889 forks source link

Build for Amazon Linux 2 is broken #9089

Open eheydrick opened 4 years ago

eheydrick commented 4 years ago

Short description

TICKET WAS: I would like to have packages for Amazon Linux 2 available in a yum repo. There are packages for Centos 7 at https://repo.powerdns.com/ but not specifically for Amazon Linux 2. Build support for Amazon Linux 2 was added in https://github.com/PowerDNS/pdns/pull/7098.

Ticket now is: we can't even consider autobuilding those packages until the build for them is fixed.

Usecase

Run pdns on Amazon Linux 2.

Habbie commented 4 years ago

Hello Eric,

adding a distribution means increasing build time, for every commit we do, by around 30 minutes. Because of that, we don't do it for distributions for which we see little demand.

We'll think about it.

(note that if you have Docker installed, the support I added in #7089 is very easy to use to build your own packages)

eheydrick commented 4 years ago

HI @Habbie, totally understand not wanting to balloon the build time. I actually did try to use the docker build but ran into a build error. Perhaps this should be a separate issue? This is what I did and the result: 

git checkout auth-4.3.0
./builder/build.sh -V 4.3.0 -m authoritative amazon-2
...
make[4]: Entering directory `/root/rpmbuild/BUILD/pdns-4.3.0/modules/bindbackend'
/bin/sh ../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I../..  -I../.. -I../.. -pthread  -I../../pdns -DLDAP_DEPRECATED  -fPIE -DPIE -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 --param ssp-buffer-size=4 -fstack-protector -g -O2 -Wall -Wextra -Wshadow -Wno-unused-parameter -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1  -m64 -mtune=generic -c -o bindbackend2.lo bindbackend2.cc
/bin/sh ../../libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I../..  -I../.. -I../.. -pthread  -I../../pdns -DLDAP_DEPRECATED  -fPIE -DPIE -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 --param ssp-buffer-size=4 -fstack-protector -g -O2 -Wall -Wextra -Wshadow -Wno-unused-parameter -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1  -m64 -mtune=generic -c -o binddnssec.lo binddnssec.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../.. -pthread -I../../pdns -DLDAP_DEPRECATED -DPIE -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 --param ssp-buffer-size=4 -fstack-protector -g -O2 -Wall -Wextra -Wshadow -Wno-unused-parameter -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -c bindbackend2.cc  -fPIC -DPIC -o .libs/bindbackend2.o
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../.. -pthread -I../../pdns -DLDAP_DEPRECATED -DPIE -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 --param ssp-buffer-size=4 -fstack-protector -g -O2 -Wall -Wextra -Wshadow -Wno-unused-parameter -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -c binddnssec.cc  -fPIC -DPIC -o .libs/binddnssec.o
In file included from ../../pdns/dnsseckeeper.hh:29:0,
                 from bindbackend2.cc:40:
/usr/include/boost/multi_index/ordered_index.hpp: In instantiation of 'boost::multi_index::detail::ordered_index<KeyFromValue, Compare, SuperMeta, TagList, Category>::reverse_iterator boost::multi_index::detail::ordered_index<KeyFromValue, Compare, SuperMeta, TagList, Category>::rbegin() [with KeyFromValue = boost::multi_index::member<BB2DomainInfo, unsigned int, &BB2DomainInfo::d_id>; Compare = std::less<unsigned int>; SuperMeta = boost::multi_index::detail::nth_layer<1, BB2DomainInfo, boost::multi_index::indexed_by<boost::multi_index::ordered_unique<boost::multi_index::member<BB2DomainInfo, unsigned int, &BB2DomainInfo::d_id> >, boost::multi_index::ordered_unique<boost::multi_index::tag<NameTag>, boost::multi_index::member<BB2DomainInfo, DNSName, &BB2DomainInfo::d_name> > >, std::allocator<BB2DomainInfo> >; TagList = boost::mpl::vector0<mpl_::na>; Category = boost::multi_index::detail::ordered_unique_tag; boost::multi_index::detail::ordered_index<KeyFromValue, Compare, SuperMeta, TagList, Category>::reverse_iterator = boost::reverse_iterator<boost::multi_index::detail::bidir_node_iterator<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::index_node_base<BB2DomainInfo, std::allocator<BB2DomainInfo> > > > > >; typename SuperMeta::type::node_type = boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::index_node_base<BB2DomainInfo, std::allocator<BB2DomainInfo> > >]':
bindbackend2.cc:1354:30:   required from here
/usr/include/boost/multi_index/ordered_index.hpp:245:63: error: call of overloaded 'make_reverse_iterator(boost::multi_index::detail::ordered_index<boost::multi_index::member<BB2DomainInfo, unsigned int, &BB2DomainInfo::d_id>, std::less<unsigned int>, boost::multi_index::detail::nth_layer<1, BB2DomainInfo, boost::multi_index::indexed_by<boost::multi_index::ordered_unique<boost::multi_index::member<BB2DomainInfo, unsigned int, &BB2DomainInfo::d_id> >, boost::multi_index::ordered_unique<boost::multi_index::tag<NameTag>, boost::multi_index::member<BB2DomainInfo, DNSName, &BB2DomainInfo::d_name> > >, std::allocator<BB2DomainInfo> >, boost::mpl::vector0<mpl_::na>, boost::multi_index::detail::ordered_unique_tag>::iterator)' is ambiguous
   reverse_iterator       rbegin(){return make_reverse_iterator(end());}
                                          ~~~~~~~~~~~~~~~~~~~~~^~~~~~~
In file included from /usr/include/boost/range/reverse_iterator.hpp:20:0,
                 from /usr/include/boost/range/rbegin.hpp:19,
                 from /usr/include/boost/range/functions.hpp:23,
                 from /usr/include/boost/range/iterator_range_core.hpp:26,
                 from /usr/include/boost/range/iterator_range.hpp:13,
                 from /usr/include/boost/range/as_literal.hpp:22,
                 from /usr/include/boost/algorithm/string/trim.hpp:19,
                 from /usr/include/boost/algorithm/string.hpp:19,
                 from bindbackend2.cc:35:
/usr/include/boost/iterator/reverse_iterator.hpp:62:43: note: candidate: boost::reverse_iterator<BidirectionalIterator> boost::make_reverse_iterator(BidirectionalIterator) [with BidirectionalIterator = boost::multi_index::detail::bidir_node_iterator<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::index_node_base<BB2DomainInfo, std::allocator<BB2DomainInfo> > > > >]
   reverse_iterator<BidirectionalIterator> make_reverse_iterator(BidirectionalIterator x)
                                           ^~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/c++/7/bits/stl_algobase.h:67:0,
                 from /usr/include/c++/7/bits/char_traits.h:39,
                 from /usr/include/c++/7/string:40,
                 from bindbackend2.cc:27:
/usr/include/c++/7/bits/stl_iterator.h:416:5: note: candidate: std::reverse_iterator<_Iterator> std::make_reverse_iterator(_Iterator) [with _Iterator = boost::multi_index::detail::bidir_node_iterator<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::index_node_base<BB2DomainInfo, std::allocator<BB2DomainInfo> > > > >]
     make_reverse_iterator(_Iterator __i)
     ^~~~~~~~~~~~~~~~~~~~~
make[4]: *** [bindbackend2.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
make[4]: Leaving directory `/root/rpmbuild/BUILD/pdns-4.3.0/modules/bindbackend'
make[3]: Leaving directory `/root/rpmbuild/BUILD/pdns-4.3.0/modules/bindbackend'
make[3]: *** [all] Error 2
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/root/rpmbuild/BUILD/pdns-4.3.0/modules'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/rpmbuild/BUILD/pdns-4.3.0'
make: *** [all] Error 2
Habbie commented 4 years ago

Perhaps this should be a separate issue?

Please! We definitely want to at least support amazon linux in that way.

pieterlexis commented 4 years ago

This is probably the combo of new compiler with old boost (or something like it). What are the versions of boost and GCC on amazon linux?

rgacogne commented 4 years ago

Looks like an ADL issue fixed in boost 1.56: https://github.com/boostorg/multi_index/commit/a9c06ce5c1f35a2b2a2c96aac986a949faddbb8a

Habbie commented 4 years ago

What are the versions of boost and GCC on amazon linux?

1.53.0, 7.3.1

Habbie commented 4 years ago

So, Amazon Linux 2 ships gcc and Boost that are mutually incompatible. The newer Boost in EPEL (1.69) is incompatible with the (newer than EL7) Python build in AL2.

I do not see a way to fix this from our side.

Habbie commented 4 years ago

I believe the following two things to be true, but I did not go through enough compile cycles to be 100% certain of anything:

@eheydrick depending on your needs (bind, geoip, ixfrdist), perhaps this allows you to build the packages you need. If you require help with that, please find us on IRC via https://www.powerdns.com/opensource.html

I'm going to rework this ticket to be about the build failures.

eheydrick commented 4 years ago

Thanks @Habbie I'll try building with clang. Another datapoint is that I'm able to run the Centos 7 auth package on AL2 and it seems to work fine but pdnsutil does not run. That's what got me looking into native built packages.

$ pdnsutil
pdnsutil: symbol lookup error: pdnsutil: undefined symbol: _ZNK5boost15program_options22error_with_option_name23substitute_placeholdersERKSs
Habbie commented 4 years ago

That's what got me looking into native built packages.

That's also what got me into building natively back when we did #7098 - but i suspect AL2 shipped an older gcc back then :(

eheydrick commented 4 years ago

Getting the same error in bind even with clang. Here's the change to the build: 

$ git diff .
diff --git a/builder-support/dockerfiles/Dockerfile.rpmbuild b/builder-support/dockerfiles/Dockerfile.rpmbuild
index 85bb0940c..50532a5fb 100644
--- a/builder-support/dockerfiles/Dockerfile.rpmbuild
+++ b/builder-support/dockerfiles/Dockerfile.rpmbuild
@@ -26,6 +26,7 @@ RUN find /pdns/builder-support/specs/ -not -name '*.spec' -exec ln -s {} /root/r
 RUN touch /var/lib/rpm/* && if $(grep -q 'release 6' /etc/redhat-release); then \
       scl enable devtoolset-7 -- builder/helpers/build-specs.sh builder-support/specs/pdns.spec; \
     else \
+      CXX=clang++ CC=clang CPP=clang-cpp \
       builder/helpers/build-specs.sh builder-support/specs/pdns.spec; \
     fi
 @ENDIF
diff --git a/builder-support/specs/pdns.spec b/builder-support/specs/pdns.spec
index 8a2a2cb19..e4c541cb3 100644
--- a/builder-support/specs/pdns.spec
+++ b/builder-support/specs/pdns.spec
@@ -26,6 +26,7 @@ Requires(postun): systemd-units
 BuildRequires: systemd
 BuildRequires: systemd-units
 BuildRequires: systemd-devel
+BuildRequires: clang

 BuildRequires: protobuf-devel
 BuildRequires: protobuf-compiler

And the result:

make[4]: Entering directory `/root/rpmbuild/BUILD/pdns-4.3.0/modules/bindbackend'
/bin/sh ../../libtool  --tag=CXX   --mode=compile clang++ -DHAVE_CONFIG_H -I. -I../..  -I../.. -I../.. -pthread  -I../../pdns -DLDAP_DEPRECATED  -fPIE -DPIE --param ssp-buffer-size=4 -fstack-protector -g -O2 -Wall -Wextra -Wshadow -Wno-unused-parameter -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1  -m64 -mtune=generic -c -o bindbackend2.lo bindbackend2.cc
/bin/sh ../../libtool  --tag=CXX   --mode=compile clang++ -DHAVE_CONFIG_H -I. -I../..  -I../.. -I../.. -pthread  -I../../pdns -DLDAP_DEPRECATED  -fPIE -DPIE --param ssp-buffer-size=4 -fstack-protector -g -O2 -Wall -Wextra -Wshadow -Wno-unused-parameter -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1  -m64 -mtune=generic -c -o binddnssec.lo binddnssec.cc
libtool: compile:  clang++ -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../.. -pthread -I../../pdns -DLDAP_DEPRECATED -DPIE --param ssp-buffer-size=4 -fstack-protector -g -O2 -Wall -Wextra -Wshadow -Wno-unused-parameter -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -c bindbackend2.cc  -fPIC -DPIC -o .libs/bindbackend2.o
libtool: compile:  clang++ -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../.. -pthread -I../../pdns -DLDAP_DEPRECATED -DPIE --param ssp-buffer-size=4 -fstack-protector -g -O2 -Wall -Wextra -Wshadow -Wno-unused-parameter -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -c binddnssec.cc  -fPIC -DPIC -o .libs/binddnssec.o
clang-7: warning: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Wunused-command-line-argument]
clang-7: warning: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Wunused-command-line-argument]
In file included from bindbackend2.cc:40:
In file included from ../../pdns/dnsseckeeper.hh:29:
/usr/include/boost/multi_index/ordered_index.hpp:245:42: error: call to 'make_reverse_iterator' is ambiguous
  reverse_iterator       rbegin(){return make_reverse_iterator(end());}
                                         ^~~~~~~~~~~~~~~~~~~~~
bindbackend2.cc:1354:23: note: in instantiation of member function 'boost::multi_index::detail::ordered_index<boost::multi_index::member<BB2DomainInfo, unsigned int, &BB2DomainInfo::d_id>, std::less<unsigned int>, boost::multi_index::detail::nth_layer<1, BB2DomainInfo, boost::multi_index::indexed_by<boost::multi_index::ordered_unique<boost::multi_index::member<BB2DomainInfo, unsigned int, &BB2DomainInfo::d_id>, mpl_::na, mpl_::na>, boost::multi_index::ordered_unique<boost::multi_index::tag<NameTag, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::multi_index::member<BB2DomainInfo, DNSName, &BB2DomainInfo::d_name>, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::allocator<BB2DomainInfo> >, boost::mpl::vector0<mpl_::na>, boost::multi_index::detail::ordered_unique_tag>::rbegin' requested here
      newid = s_state.rbegin()->d_id+1;
                      ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/7/../../../../include/c++/7/bits/stl_iterator.h:416:5: note: candidate function [with _Iterator = boost::multi_index::detail::bidir_node_iterator<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::index_node_base<BB2DomainInfo, std::allocator<BB2DomainInfo> > > > >]
    make_reverse_iterator(_Iterator __i)
    ^
/usr/include/boost/iterator/reverse_iterator.hpp:62:43: note: candidate function [with BidirectionalIterator = boost::multi_index::detail::bidir_node_iterator<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::ordered_index_node<boost::multi_index::detail::index_node_base<BB2DomainInfo, std::allocator<BB2DomainInfo> > > > >]
  reverse_iterator<BidirectionalIterator> make_reverse_iterator(BidirectionalIterator x)
                                          ^
1 error generated.
Habbie commented 4 years ago

ugh, then I must have tested the clang build without bind accidentally. Sorry :(