search

randombit / botan

Cryptography Toolkit
https://botan.randombit.net
BSD 2-Clause "Simplified" License
2.59k stars 569 forks source link

Some build flags are not passed to link-time optimizer when building with `-flto=auto` #4196

Closed mrc0mmand closed 4 months ago

mrc0mmand commented 4 months ago

While preparing another batch of botan2 updates in Fedora, I noticed a failing check in annocheck:

$ annocheck /usr/bin/botan 
annocheck: Version 12.46.
Hardened: botan: FAIL: stack-prot test because insufficient protection enabled (based on contents of DW_AT_producer string) 
Hardened: Rerun annocheck with --verbose to see more information on the tests.
Hardened: botan: Overall: FAIL.

Making annocheck a bit more verbose shows where this information comes from:

$ annocheck -v -v /usr/bin/botan                                                                                                                                                                                    
annocheck: Version 12.46.  
...
Hardened: /usr/bin/botan: DW_AT_producer = GNU GIMPLE 13.2.1 20230728 (Red Hat 13.2.1-1) -mno-omit-leaf-frame-pointer -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fasynchronous-unwind-tables
 -fcf-protection=full -fPIE -fstack-protector -fltrans -fplugin=annobin.                                  
Hardened: /usr/bin/botan: info: record producer: Gimple version: 13 source: DW_AT_producer string.                                                                                                                  
Hardened: /usr/bin/botan: info: set binary producer to Gimple version 13.                                                                                                                                           
Hardened: /usr/bin/botan: PASS: pic test because option found in DW_AT_producer string  (source: DW_AT_producer string)                                                                                             
Hardened: /usr/bin/botan: FAIL: stack-prot test because insufficient protection enabled (based on contents of DW_AT_producer string) (source: DW_AT_producer string)                                                
Hardened: /usr/bin/botan: info: For more information visit: https://sourceware.org/annobin/annobin.html/Test-stack-prot.html                                                    
...

With this information I was also able to reproduce this with a reduced set of build flags on the latest main (i.e. botan3):

$ CXXFLAGS="-fstack-protector-strong -O2 -g -flto=auto" ./configure.py
...
$ make -j $(nproc)
...
$ objdump -Wi botan | grep DW_AT_producer 
    <d>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <56f57>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <9c973>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <d1914>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <115fca>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <15b8a0>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <19fff1>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <1e7302>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <22ac33>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <273833>   DW_AT_producer    : (indirect string, offset: 0x1d2): GNU GIMPLE 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector -fltrans
    <2b828c>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <2c392c>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <2d7d79>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <2e3431>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <2f9a17>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <30b7ac>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <336f33>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <35509e>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <372658>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
    <3747a3>   DW_AT_producer    : (indirect string, offset: 0x5280): GNU C++20 13.2.1 20240316 (Red Hat 13.2.1-7) -m64 -mtune=generic -march=x86-64 -g -O2 -std=c++20 -fstack-protector -fstack-protector-strong -flto
...

Notice that -fstack-protector-strong is missing from all GNU GIMPLE lines, causing annocheck to fail. When building with the full set of Fedora build flags (rpm --eval "%{build_cxxflags}") I noticed that there are other flags missing as well (-fexception, -ffat-lto-objects, ...).

I've never encountered this error from annocheck before, so I'm not really sure who to blame here.

randombit commented 4 months ago

A possible reason for this is that CXXFLAGS is reflected into the compilations of individual files but not in the link. You could achieve this by setting LDFLAGS similarly.

The behavior matches my current understanding of the convention, but since -flto is required at compilation and link time, it seems you're expecting CXXFLAGS is also used for the link step.

mrc0mmand commented 4 months ago

I was confused why this doesn't seem to be an issue for other packages, and it looks like some/other build systems do what you describe automagically, i.e.:

$ meson init -l cpp
...
$ CXXFLAGS="-O2 -g -fstack-protector-strong -flto=auto" meson setup build
...
$ ninja -C build -v
ninja: Entering directory `build'
[1/2] c++ -Imeson.p -I. -I.. -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Wpedantic -std=c++14 -O0 -g -O2 -g -fstack-protector-strong -flto=auto -MD -MQ meson.p/meson.cpp.o -MF meson.p/meson.cpp.o.d -o meson.p/meson.cpp.o -c ../meson.cpp
[2/2] c++  -o meson meson.p/meson.cpp.o -Wl,--as-needed -Wl,--no-undefined -O2 -g -fstack-protector-strong -flto=auto
$ objdump -Wi build/meson | grep DW_AT_producer
    <d>   DW_AT_producer    : (indirect string, offset: 0): GNU GIMPLE 13.3.1 20240522 (Red Hat 13.3.1-1) -mtune=generic -march=x86-64 -g -g -O0 -O2 -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector-strong -fltrans
    <fe>   DW_AT_producer    : (indirect string, offset: 0x23a): GNU C++14 13.3.1 20240522 (Red Hat 13.3.1-1) -mtune=generic -march=x86-64 -g -g -O0 -O2 -std=c++14 -fstack-protector-strong -flto
$ grep . CMakeLists.txt foo.cxx 
CMakeLists.txt:cmake_minimum_required (VERSION 3.7)
CMakeLists.txt:project (Foo)
CMakeLists.txt:add_executable(Foo foo.cxx)
foo.cxx:#include <iostream>
foo.cxx:int main (void) {
foo.cxx:    std::cout << "Hello" << std::endl;
foo.cxx:    return 0;
foo.cxx:}
$ mkdir build && cd build
$ CXXFLAGS="-O2 -g -fstack-protector-strong -flto=auto" cmake ..
...
$ make VERBOSE=1
/usr/bin/cmake -S/home/mrc0mmand/tmp/cmake -B/home/mrc0mmand/tmp/cmake/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/mrc0mmand/tmp/cmake/build/CMakeFiles /home/mrc0mmand/tmp/cmake/build//CMakeFiles/progress.marks
make  -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/home/mrc0mmand/tmp/cmake/build'
make  -f CMakeFiles/Foo.dir/build.make CMakeFiles/Foo.dir/depend
make[2]: Entering directory '/home/mrc0mmand/tmp/cmake/build'
cd /home/mrc0mmand/tmp/cmake/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/mrc0mmand/tmp/cmake /home/mrc0mmand/tmp/cmake /home/mrc0mmand/tmp/cmake/build /home/mrc0mmand/tmp/cmake/build /home/mrc0mmand/tmp/cmake/build/CMakeFiles/Foo.dir/DependInfo.cmake "--color="
make[2]: Leaving directory '/home/mrc0mmand/tmp/cmake/build'
make  -f CMakeFiles/Foo.dir/build.make CMakeFiles/Foo.dir/build
make[2]: Entering directory '/home/mrc0mmand/tmp/cmake/build'
[ 50%] Building CXX object CMakeFiles/Foo.dir/foo.cxx.o
/usr/bin/c++   -O2 -g -fstack-protector-strong -flto=auto -MD -MT CMakeFiles/Foo.dir/foo.cxx.o -MF CMakeFiles/Foo.dir/foo.cxx.o.d -o CMakeFiles/Foo.dir/foo.cxx.o -c /home/mrc0mmand/tmp/cmake/foo.cxx
[100%] Linking CXX executable Foo
/usr/bin/cmake -E cmake_link_script CMakeFiles/Foo.dir/link.txt --verbose=1
/usr/bin/c++ -O2 -g -fstack-protector-strong -flto=auto CMakeFiles/Foo.dir/foo.cxx.o -o Foo 
make[2]: Leaving directory '/home/mrc0mmand/tmp/cmake/build'
[100%] Built target Foo
make[1]: Leaving directory '/home/mrc0mmand/tmp/cmake/build'
/usr/bin/cmake -E cmake_progress_start /home/mrc0mmand/tmp/cmake/build/CMakeFiles 0
$ objdump -Wi Foo | grep DW_AT_producer
    <d>   DW_AT_producer    : (indirect string, offset: 0): GNU GIMPLE 13.3.1 20240522 (Red Hat 13.3.1-1) -mtune=generic -march=x86-64 -g -g -O2 -O2 -fno-openmp -fno-openacc -fno-pie -fcf-protection=none -fstack-protector-strong -fltrans
    <225>   DW_AT_producer    : (indirect string, offset: 0xf8f): GNU C++17 13.3.1 20240522 (Red Hat 13.3.1-1) -mtune=generic -march=x86-64 -g -O2 -fstack-protector-strong -flto
randombit commented 4 months ago

Can you try the patch in #4200?

randombit commented 4 months ago

This has been fixed now in both release-2 and master branches, so will be picked up in the next release. Thanks for reporting this

mrc0mmand commented 4 months ago

Thank you! I'm preparing a botan3 package for Fedora, so the patch for the 3.x (master) branch is much appreciated.

randombit commented 4 months ago

@mrc0mmand That's great to hear. Don't know if you've seen them already but there are some notes for distributors that may be helpful.