Broken compilation with gcc v6.x if CXXFLAGS env var is set

[chros73]

Couple of our users reported that they can't compile rtorrent-ps-ch on newer distros (not just on a particular one), one of them, @colinhd8 even came up with an unexpected fix: unset the CXXFLAGS env var in our script. The only common thing amongst those distros is the gcc compiler version, which is v6.x .

Since I wasn't sure where the actual issue relies (in our automagic function, other part of our script, etc.) I've created a minimal build script on fresh install of Debian 9 (using gcc v6.3.0) to see whether rtorrent can be built or not.

Conclusion: if the CXXFLAGS env var is set (even when it's empty string) building "fails" at the end of compilation process only when rtorrent tries to link libtorrent.so :

../rak/priority_queue_default.h: In destructor ‘rak::priority_item::~priority_item()’:
../rak/priority_queue_default.h:56:95: warning: throw will always call terminate() [-Wterminate]
       throw torrent::internal_error("priority_item::~priority_item() called on a queued item.");
                                                                                               ^
../rak/priority_queue_default.h:56:95: note: in C++11 destructors default to noexcept
mv -f .deps/main.Tpo .deps/main.Po
/bin/bash ../libtool  --tag=CXX   --mode=link g++  -g -DDEBUG -Wall   -I/home/test/lib/rtorrent-0.9.7-1.5.1/include  -I/home/test/lib/rtorrent-0.9.7-1.5.1/include   -L/home/test/lib/rtorrent-0.9.7-1.5.1/lib -o rtorrent main.o libsub_root.a ui/libsub_ui.a core/libsub_core.a display/libsub_display.a input/libsub_input.a rpc/libsub_rpc.a utils/libsub_utils.a  -lncursesw -lcppunit -L/home/test/lib/rtorrent-0.9.7-1.5.1/lib -lcurl -ltorrent -lpthread -L/home/test/lib/rtorrent-0.9.7-1.5.1/lib   -lxmlrpc_server -lxmlrpc  -lxmlrpc_xmlparse -lxmlrpc_xmltok -lxmlrpc_util -lpthread 
libtool: link: g++ -g -DDEBUG -Wall -I/home/test/lib/rtorrent-0.9.7-1.5.1/include -I/home/test/lib/rtorrent-0.9.7-1.5.1/include -o rtorrent main.o  -L/home/test/lib/rtorrent-0.9.7-1.5.1/lib libsub_root.a ui/libsub_ui.a core/libsub_core.a display/libsub_display.a input/libsub_input.a rpc/libsub_rpc.a utils/libsub_utils.a -lncursesw -lcppunit /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libcurl.so /home/test/lib/rtorrent-0.9.7-1.5.1/lib/libtorrent.so -lxmlrpc_server -lxmlrpc -lxmlrpc_xmlparse -lxmlrpc_xmltok -lxmlrpc_util -lpthread -Wl,-rpath -Wl,/home/test/lib/rtorrent-0.9.7-1.5.1/lib -Wl,-rpath -Wl,/home/test/lib/rtorrent-0.9.7-1.5.1/lib
/home/test/lib/rtorrent-0.9.7-1.5.1/lib/libtorrent.so: undefined reference to `torrent::PeerConnection<(torrent::Download::ConnectionType)1>::offer_chunk()'
/home/test/lib/rtorrent-0.9.7-1.5.1/lib/libtorrent.so: undefined reference to `torrent::PeerConnection<(torrent::Download::ConnectionType)0>::offer_chunk()'
collect2: error: ld returned 1 exit status

The same can be achieved earlier when rtorrent is configured with xmlrpc suppport: this time the checking macro fails at the biginning of building process of rtorrent: resulting the very same error message as above in config.log.

In summary:

• other env vars we use aren't affected by this only CXXFLAGS
• it's distro independent
• it only depends on the version of gcc: 4.x , 5.x work fine (I'm not sure about 7.x , like in newer Arch Linux)
• both master and feature-bind branches are affected
• strangly enough 0.9.6/0.13.6 is NOT affected by this

This is a really nasty bug, e.g. in Gentoo such env vars can be defined system wide in /etc/make.conf. And that means no additional flags can be set in this way. I quickly went through the configure.ac and all the m4 macros in scripts dir but I didn't see anything unusual. More than that almost all the marcos are identical in rtorrent and libtorrent projects: so I'm not sure at all that rtorrent isn't affected by this, since libtorrent builds fine seemingly until it's not linked!

Maybe it would be worth to modify Travis scripts to:

• include newer compilers: 4.x, 5.x, 6.x, 7x
• export these empty env vars at the beginning of the script

Example distros that are affected:

• Debian 9, gcc 6.3
• Fedora 24 x64 GCC:6.3.1 20161221 (Red Hat 6.3.1-1) (GCC)
• CentOs 7 x64 GCC:gcc version 6.2.1 20160916 (Red Hat 6.2.1-3) (GCC)
• Ubuntu Server 17.04, gcc 6.3.0
• ViodLinux gcc 6.3

[rakshasa]

Looks like a new compiler warning, which will break the compile if using the '-Werror' flag.

-Wterminate (enabled by default) warns when a throw will immediate result in a call to terminate like in an noexcept function. In particular it will warn when something is thrown from a C++11 destructor since they default to noexcept, unlike in C++98 (GCC6 defaults to -std=gnu++14).

I guess adding 'no-terminate' warning flag would fix that.

[chros73]

Thanks for the quick reply. I'm not user whether I understand, if I add this flag to CXXFLAGS export CXXFLAGS="$CFLAGS -no-terminate" (note CFLAGS is empty) then this error occures at the beginning of building libtorrent with the above mentioned minimal script, and compilation stops:

checking dependency style of g++... gcc3
checking whether byte ordering is bigendian... no
checking whether g++ supports C++11 features by default... no
checking whether g++ supports C++11 features with -std=c++11... no
checking whether g++ supports C++11 features with -std=c++0x... no
configure: error: *** A compiler with support for C++11 language features is required.

[chros73]

A. Reply: I was wrong, and you were right: I thought it fails at linking time, but it fails to compile both of the projects! (I guess our build script needs to be improved to signal the user at which step it failed.)

I guess adding 'no-terminate' warning flag would fix that.

No, it doesn't. I found this page, so I was wrong about about the suggested flag, it should be -Wno-terminate, but it still fails at the very same way. I tried out this way:

export CFLAGS="-Wno-terminate ${CFLAGS}"
export CXXFLAGS="${CFLAGS}"

What I don't understand is why it fails when an unharmful flag is given, like -pipe.

What makes this strange bug even more interesting that there are reports:

• it works fine with gcc v.7.1.x ! (on Slackware and Fedora)

B. I found the issue: It's the combination of the following (there's a logical AND between them):

• any value of CXXFLAGS is set and
• gcc version is 6.x (7???) and
• current version (master/feature-bind) of rtorrent/libtorrent is used and
• missing -O2 flag and
• using make -jx (using simple make compiles fine!)

So, setting -O2 flag in the above mentioned script fixes the issue with gcc v6.x.

export CFLAGS="-O2 ${CFLAGS}"
export CXXFLAGS="${CFLAGS}"

Please note the the mentioned script uses make -j4 for all of the project.

I found out this by comparing the working and failing generated Makefiles , and the difference was the missing -O2 flag:

< CFLAGS = -pthread  -pipe
---
> CFLAGS = -pthread
263c263
< CXXFLAGS = -pthread  -pipe  -g -DDEBUG -Wall -fvisibility=hidden
---
> CXXFLAGS = -pthread  -g -O2 -g -DDEBUG -Wall -fvisibility=hidden

C. Solution? It's definitely a problem with libtorrent/rtorrent, because all the other libraries (c-ares, curl, xmlrpc-c) compiles fine with these setting.

If you can point out where I shall look I'll try to fix it.

[chros73]

D. Solution: OK, since 0.9.6/0.13.6 compiled fine, it was easy to figure it out what's the problem:

• both projects are affected by this

This was removed from scripts/common.m4 (from both projects):

AC_DEFUN([TORRENT_CHECK_CXXFLAGS], [

  AC_MSG_CHECKING([for user-defined CXXFLAGS])

  if test -n "$CXXFLAGS"; then
    AC_MSG_RESULT([user-defined "$CXXFLAGS"])
  else
    CXXFLAGS="-O2 -Wall"
    AC_MSG_RESULT([default "$CXXFLAGS"])
  fi
])

and the corresponding entry from configure.ac (from both projects):

TORRENT_CHECK_CXXFLAGS

The result is with gcc 6.x (but probably it's a libtool "feature"):

• if CXXFLAGS is set but empty then it treats it as a user defined value, and still clears out the -O2 flag:

  export CFLAGS="${CFLAGS}"
  export CXXFLAGS="${CFLAGS}"

So, I suggest to put it back, it won't hurt anything. If you want I can create a pull request for it for both repos, or you can quickly add to the feature-bind branches.

Ps: The good news is:

• master branches are really stable, even with my patches :)
  • they compile on bunch of different distros and run fine

And let me thank You for all the effort you put in the last ~13 years in this very versatile client! Thank You!

[rakshasa]

Please do.

I'm still (slowly) working on the rtorrent-vagrant project to make some automated testing of different network configurations done.

[chros73]

Please do.

No worries. I'll do it during the weekend, carefully looking after other changes in scripts folder (maybe only the -O2 flag has to be added).

I'm still (slowly) working on the rtorrent-vagrant project to make some automated testing of different network configurations done.

:) I can imagine that it's a pain in the *** :) Don't give up, take your time!

[chros73]

I have updated the fix:

• let the building environment (autotools) define what flags are used only if CFLAGS/CXXFLAGS is empty

The reason behind this is that set but empty flags shouldn't be specified at all.

[voron]

I still have this bug on my gentoo with GCC 6.4 and rtorrent 0.9.7/libtorrent 0.13.7

libtool: link: x86_64-pc-linux-gnu-g++ -mcx16 -mfxsr -msahf -mtune=generic -pipe -g -g -DDEBUG -Wall -pthread -I/usr/include -Wl,-O1 -o rtorrent main.o  -Wl,--as-needed libsub_root.a ui/libsub_ui.a core/libsub_core.a display/libsub_display.a input/libsub_input.a rpc/libsub_rpc.a utils/libsub_utils.a -lncursesw -lcppunit -lcurl -ltorrent -L/usr/lib64 -lxmlrpc_server -lxmlrpc -lxml2 -lz -lm -ldl -lxmlrpc_util -lpthread -pthread
/usr/lib64/libtorrent.so: undefined reference to torrent::PeerConnection<(torrent::Download::ConnectionType)1>::offer_chunk()
/usr/lib64/libtorrent.so: undefined reference to torrent::PeerConnection<(torrent::Download::ConnectionType)0>::offer_chunk()
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:554: rtorrent] Error 1

I had tried single thread build, but it changed nothing.

[chros73]

It indeed looks the same problem. Did you have the -O2 flag as in the point B. of this comment?

Just for out of curiosity: can you compile rtorrent-ps-ch with it's build script? (I never worked with Gentoo, so try to make sure that all the package reqs are fulfilled.)

[voron]

Did you have the -O2 flag as in the point B. of this comment?

No, I didn't have it. I added it, remerged libtorrent 0.13.7 and rtorrent 0.9.7 fine, now rtorrent works again.