Build fails with LTO

[eli-schwartz]

/usr/bin/bash ../libtool  --tag=CXX   --mode=link x86_64-pc-linux-gnu-g++  -march=native -fstack-protector-all -O2 -pipe -fdiagnostics-color=always -frecord-gcc-switches -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-clash-protection -flto=4 -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing  -Wformat -Werror=format-security -DNDEBUG -Wall -isysroot /        -Wl,-O1 -Wl,--as-needed -flto=4 -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing -Wl,--defsym=__gentoo_check_ldflags__=0 -Wl,--sysroot,/ -o rtorrent main.o libsub_root.a ui/libsub_ui.a core/libsub_core.a display/libsub_display.a input/libsub_input.a rpc/libsub_rpc.a utils/libsub_utils.a  -lncursesw -ltinfow  -lcppunit  -lcurl -ltorrent 
libtool: link: x86_64-pc-linux-gnu-g++ -march=native -fstack-protector-all -O2 -pipe -fdiagnostics-color=always -frecord-gcc-switches -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-clash-protection -flto=4 -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing -Wformat -Werror=format-security -DNDEBUG -Wall -isysroot / -Wl,-O1 -flto=4 -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing -Wl,--defsym=__gentoo_check_ldflags__=0 -Wl,--sysroot -Wl,/ -o rtorrent main.o  -Wl,--as-needed libsub_root.a ui/libsub_ui.a core/libsub_core.a display/libsub_display.a input/libsub_input.a rpc/libsub_rpc.a utils/libsub_utils.a -lncursesw -ltinfow -lcppunit -lcurl -ltorrent
utils/directory.h:46:8: error: type 'struct directory_entry' violates the C++ One Definition Rule [-Werror=odr]
   46 | struct directory_entry {
      |        ^
input/./../utils/directory.h:46:8: note: a different type is defined in another translation unit
   46 | struct directory_entry {
      |        ^
utils/directory.h:51:23: note: the first difference of corresponding definitions is field 'd_fileno'
   51 |   uint32_t            d_fileno;
      |                       ^
input/./../utils/directory.h:51:23: note: a field with different name is defined in another translation unit
   51 |   uint32_t            d_fileno;
      |                       ^
lto1: some warnings being treated as errors
lto-wrapper: fatal error: x86_64-pc-linux-gnu-g++ returned 1 exit status

Full logs: build.log Downstream report: https://bugs.gentoo.org/861848

[kannibalox]

Correct me if I'm misunderstanding, but utils/directory.h:46:8: and input/./../utils/directory.h:46:8 are the exact same file, so it sounds like whatever build system gentoo is using is generating duplicate definitions for itself in the libsub_*.a files. Using the documented build process with -Werror=odr compiles perfectly fine for me.

I doubt rTorrent will ever officially support gentoo's build system, but even if I wanted to help unofficially, there's not any information on how to reproduce.

[eli-schwartz]

Correct me if I'm misunderstanding, but utils/directory.h:46:8: and input/./../utils/directory.h:46:8 are the exact same file, so it sounds like whatever build system gentoo is using is generating duplicate definitions for itself in the libsub_*.a files. Using the documented build process with -Werror=odr compiles perfectly fine for me.

I doubt rTorrent will ever officially support gentoo's build system, but even if I wanted to help unofficially, there's not any information on how to reproduce.

The best way to reproduce would be to open the attachment titled "full logs" and check for information on how to reproduce.

In short, Gentoo is not using an alternative build system, there's nothing specific to Gentoo here.

It's just running configure and make.

I would be happy to provide any additional details needed but if we can't agree on whether or not the official configure and make scripts were used then I'm not sure how much anything I say matters.

[kannibalox]

So there's no set of emerge commands I could run to quickly reproduce on a gentoo docker image or anything? I spent some time looking at the log, and tried docker run gentoo/stage3 "emerge --sync && emerge '=net-p2p/rtorrent-0.9.8-r2::gentoo'", but didn't get any error. Trying to adapt the configure command to my system would be possible but also seems pretty tedious (not to mention trying to match toolset versions), so if there's an easier way it would be greatly appreciated.

[eli-schwartz]

In the Gentoo container, if you add these flags to /etc/portage/make.conf as part of CFLAGS / CXXFLAGS / LDFLAGS, it should demonstrate the issue:

-flto -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing

Note: -flto is needed to produce LTO code generation, and the -Werror=* flags cause the link-time "lto-wrapper" based codegen to fatally error when safety constraints are violated.

In principle it should reproduce with a plain configure and make, when using those flags. I hastily tried that with rtorrent from a git clone, but it didn't appear to compile due to libtorrent version differences and I didn't have time right then to chase that down.

"Probably", using those flags on your own system with your own toolset versions and whatever configure command you usually use, would be enough too.

[kannibalox]

I gave it enough of an effort to ensure that it's really LTO shenanigans at play, rather than a true ODR violation. It doesn't seem insurmountable, so maybe I'll take another shot later, but all the solutions I found didn't seem to be worth the increase in complexity.

[stickz]

@eli-schwartz I resolved this issue in my version of rTorrent. The fix was very simplistic. It will build with LTO now on Gentoo Linux. https://github.com/stickz/rtorrent/commit/6332d392318079fd3fd5504a549d4f992ce50005

Just need to run git clone if you're interested and follow the build instructions: git clone https://github.com/stickz/rtorrent https://github.com/stickz/rtorrent

[eli-schwartz]

Cool :) is there a PR including that commit?

[stickz]

Cool :) is there a PR including that commit?

No, I had to rebuild the repo to get the commit history. There's just a commit.

[eli-schwartz]

Thanks for the PR and for the new release -- I've pinged our package maintainer to take a look at packaging the update.

[stickz]

Thanks for the PR and for the new release -- I've pinged our package maintainer to take a look at packaging the update.

@eli-schwartz You'll need to backport a couple patches that have already been merged into upstream. This will fix the builds for Gentoo Linux and create a stable experience. https://github.com/rakshasa/rtorrent/commit/c1d076f1822752d4956a884a9aa9e6b0026b11f3 and https://github.com/rakshasa/rtorrent/commit/38b39bdafc9edd7e7e72672e6fcbd397b6c2cab8

If you create two patches for these, the v0.10.0 release will be more stable than v0.9.8.

[eli-schwartz]

@eli-schwartz You'll need to backport a couple patches that have already been merged into upstream. This will fix the builds for Gentoo Linux and create a stable experience. c1d076f and 38b39bd

The second commit looks unnecessary except inasmuch as running autoreconf -fi for rtorrent is, I guess, not supposed to depend on installing additional maintainer dependencies such as autoconf-archive. It is possible to ship those macros only in release tarballs and not in git, depending on how a project maintainer handles the usual workflow.

[eli-schwartz]

@stickz the first commit looks unnecessary too. rtorrent is a program, not a libtool library. The commit adds LT_INIT and invokes the libtool machinery to compile and link the /usr/bin/rtorrent binary instead of directly running the compiler -- dependencies are anyways fetched via pkg-config.

I don't see how using LT_INIT can fix static building. The resulting ./configure --enable-static --enable-shared options aren't even utilized for building binaries, only for building libraries, which would matter for libtorrent, not rtorrent.

It's a bit slower to build using the libtool proxy and I don't see any benefit to using it here. And if I wanted a static build of rtorrent, I would simply build libtorrent with --enable-static --disable-shared and then compile rtorrent with LDFLAGS="-static".