compile failed with nginx when cmake build brotli return plugin needed to handle lto object

[kn007]

cmake version 3.27.7 nginx 1.25.3 gcc (GCC) 10.3.1 20210422 (Red Hat 10.3.1-1)

export CFLAGS="-m64 -march=native -mtune=native -Ofast -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections"
export LDFLAGS="-m64 -Wl,-s -Wl,-Bsymbolic -Wl,--gc-sections"

-m64 -Wl,-s -Wl,-Bsymbolic -Wl,--gc-sections -fPIC -ljemalloc -lrt -flto=4 -fuse-ld=gold -ldl -luring -lpthread -lpthread -lcrypt -L/data/src/ngx_brotli/deps/brotli/c/../out -lbrotlienc -lbrotlicommon -lm -lpcre2-8 /data/src/openssl-1.1.1w/.openssl/lib/libssl.a /data/src/openssl-1.1.1w/.openssl/lib/libcrypto.a -ldl -lpthread /data/src/zlib-cf/libz.a -latomic_ops \
-Wl,-E
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:555: error: undefined reference to 'BrotliEncoderCreateInstance'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:563: error: undefined reference to 'BrotliEncoderSetParameter'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:572: error: undefined reference to 'BrotliEncoderSetParameter'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:628: error: undefined reference to 'BrotliEncoderDestroyInstance'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:452: error: undefined reference to 'BrotliEncoderIsFinished'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:492: error: undefined reference to 'BrotliEncoderCompressStream'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:423: error: undefined reference to 'BrotliEncoderHasMoreOutput'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:425: error: undefined reference to 'BrotliEncoderTakeOutput'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:463: error: undefined reference to 'BrotliEncoderCompressStream'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:628: error: undefined reference to 'BrotliEncoderDestroyInstance'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:437: error: undefined reference to 'BrotliEncoderIsFinished'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:628: error: undefined reference to 'BrotliEncoderDestroyInstance'
/data/src/ngx_brotli/filter/ngx_http_brotli_filter_module.c:628: error: undefined reference to 'BrotliEncoderDestroyInstance'
collect2: error: ld returned 1 exit status
make[1]: *** [objs/Makefile:411: objs/nginx] Error 1

==================================================

git clone --recurse-submodules https://github.com/google/ngx_brotli.git
cd ngx_brotli/deps/brotli
mkdir out && cd out
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
cmake --build . --config Release --target brotlienc

[  3%] Building C object CMakeFiles/brotlicommon.dir/c/common/constants.c.o
[  6%] Building C object CMakeFiles/brotlicommon.dir/c/common/context.c.o
[ 10%] Building C object CMakeFiles/brotlicommon.dir/c/common/dictionary.c.o
[ 13%] Building C object CMakeFiles/brotlicommon.dir/c/common/platform.c.o
[ 17%] Building C object CMakeFiles/brotlicommon.dir/c/common/shared_dictionary.c.o
[ 20%] Building C object CMakeFiles/brotlicommon.dir/c/common/transform.c.o
[ 24%] Linking C static library libbrotlicommon.a
/bin/ar: CMakeFiles/brotlicommon.dir/c/common/constants.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlicommon.dir/c/common/context.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlicommon.dir/c/common/dictionary.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlicommon.dir/c/common/platform.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlicommon.dir/c/common/shared_dictionary.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlicommon.dir/c/common/transform.c.o: plugin needed to handle lto object
/bin/ranlib: libbrotlicommon.a(constants.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlicommon.a(context.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlicommon.a(dictionary.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlicommon.a(platform.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlicommon.a(shared_dictionary.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlicommon.a(transform.c.o): plugin needed to handle lto object
[ 24%] Built target brotlicommon
[ 27%] Building C object CMakeFiles/brotlienc.dir/c/enc/backward_references.c.o
[ 31%] Building C object CMakeFiles/brotlienc.dir/c/enc/backward_references_hq.c.o
[ 34%] Building C object CMakeFiles/brotlienc.dir/c/enc/bit_cost.c.o
[ 37%] Building C object CMakeFiles/brotlienc.dir/c/enc/block_splitter.c.o
[ 41%] Building C object CMakeFiles/brotlienc.dir/c/enc/brotli_bit_stream.c.o
[ 44%] Building C object CMakeFiles/brotlienc.dir/c/enc/cluster.c.o
[ 48%] Building C object CMakeFiles/brotlienc.dir/c/enc/command.c.o
[ 51%] Building C object CMakeFiles/brotlienc.dir/c/enc/compound_dictionary.c.o
[ 55%] Building C object CMakeFiles/brotlienc.dir/c/enc/compress_fragment.c.o
[ 58%] Building C object CMakeFiles/brotlienc.dir/c/enc/compress_fragment_two_pass.c.o
[ 62%] Building C object CMakeFiles/brotlienc.dir/c/enc/dictionary_hash.c.o
[ 65%] Building C object CMakeFiles/brotlienc.dir/c/enc/encode.c.o
[ 68%] Building C object CMakeFiles/brotlienc.dir/c/enc/encoder_dict.c.o
[ 72%] Building C object CMakeFiles/brotlienc.dir/c/enc/entropy_encode.c.o
[ 75%] Building C object CMakeFiles/brotlienc.dir/c/enc/fast_log.c.o
[ 79%] Building C object CMakeFiles/brotlienc.dir/c/enc/histogram.c.o
[ 82%] Building C object CMakeFiles/brotlienc.dir/c/enc/literal_cost.c.o
[ 86%] Building C object CMakeFiles/brotlienc.dir/c/enc/memory.c.o
[ 89%] Building C object CMakeFiles/brotlienc.dir/c/enc/metablock.c.o
[ 93%] Building C object CMakeFiles/brotlienc.dir/c/enc/static_dict.c.o
[ 96%] Building C object CMakeFiles/brotlienc.dir/c/enc/utf8_util.c.o
[100%] Linking C static library libbrotlienc.a
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/backward_references.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/backward_references_hq.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/bit_cost.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/block_splitter.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/brotli_bit_stream.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/cluster.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/command.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/compound_dictionary.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/compress_fragment.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/compress_fragment_two_pass.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/dictionary_hash.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/encode.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/encoder_dict.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/entropy_encode.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/fast_log.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/histogram.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/literal_cost.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/memory.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/metablock.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/static_dict.c.o: plugin needed to handle lto object
/bin/ar: CMakeFiles/brotlienc.dir/c/enc/utf8_util.c.o: plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(backward_references.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(backward_references_hq.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(bit_cost.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(block_splitter.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(brotli_bit_stream.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(cluster.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(command.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(compound_dictionary.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(compress_fragment.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(compress_fragment_two_pass.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(dictionary_hash.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(encode.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(encoder_dict.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(entropy_encode.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(fast_log.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(histogram.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(literal_cost.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(memory.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(metablock.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(static_dict.c.o): plugin needed to handle lto object
/bin/ranlib: libbrotlienc.a(utf8_util.c.o): plugin needed to handle lto object
[100%] Built target brotlienc

Maybe it's because plugin needed to handle lto object ?

[kn007]

temp fix: change CMAKE_AR & CMAKE_RANLIB value from ngx_brotli/deps/brotli/out/CMakeFiles/3.27.7/CMakeCCompiler.cmake ar to gcc-ar, ranlib to gcc-ranlib

ar & ranlib version 2.30-108.el8_5.1 gcc-ar & gcc-ranlib version 2.35-8.el8_5.6

> cmake --build . --config Release --target brotlienc                                                                                                                       ed738e8 [ed738e8] untracked
-- Build type is 'Release'
-- Compiler is not EMSCRIPTEN
-- Configuring done (0.0s)
-- Generating done (0.1s)
-- Build files have been written to: /data/src/ngx_brotli/deps/brotli/out
[  3%] Building C object CMakeFiles/brotlicommon.dir/c/common/constants.c.o
[  6%] Building C object CMakeFiles/brotlicommon.dir/c/common/context.c.o
[ 10%] Building C object CMakeFiles/brotlicommon.dir/c/common/dictionary.c.o
[ 13%] Building C object CMakeFiles/brotlicommon.dir/c/common/platform.c.o
[ 17%] Building C object CMakeFiles/brotlicommon.dir/c/common/shared_dictionary.c.o
[ 20%] Building C object CMakeFiles/brotlicommon.dir/c/common/transform.c.o
[ 24%] Linking C static library libbrotlicommon.a
[ 24%] Built target brotlicommon
[ 27%] Building C object CMakeFiles/brotlienc.dir/c/enc/backward_references.c.o
[ 31%] Building C object CMakeFiles/brotlienc.dir/c/enc/backward_references_hq.c.o
[ 34%] Building C object CMakeFiles/brotlienc.dir/c/enc/bit_cost.c.o
[ 37%] Building C object CMakeFiles/brotlienc.dir/c/enc/block_splitter.c.o
[ 41%] Building C object CMakeFiles/brotlienc.dir/c/enc/brotli_bit_stream.c.o
[ 44%] Building C object CMakeFiles/brotlienc.dir/c/enc/cluster.c.o
[ 48%] Building C object CMakeFiles/brotlienc.dir/c/enc/command.c.o
[ 51%] Building C object CMakeFiles/brotlienc.dir/c/enc/compound_dictionary.c.o
[ 55%] Building C object CMakeFiles/brotlienc.dir/c/enc/compress_fragment.c.o
[ 58%] Building C object CMakeFiles/brotlienc.dir/c/enc/compress_fragment_two_pass.c.o
[ 62%] Building C object CMakeFiles/brotlienc.dir/c/enc/dictionary_hash.c.o
[ 65%] Building C object CMakeFiles/brotlienc.dir/c/enc/encode.c.o
[ 68%] Building C object CMakeFiles/brotlienc.dir/c/enc/encoder_dict.c.o
[ 72%] Building C object CMakeFiles/brotlienc.dir/c/enc/entropy_encode.c.o
[ 75%] Building C object CMakeFiles/brotlienc.dir/c/enc/fast_log.c.o
[ 79%] Building C object CMakeFiles/brotlienc.dir/c/enc/histogram.c.o
[ 82%] Building C object CMakeFiles/brotlienc.dir/c/enc/literal_cost.c.o
[ 86%] Building C object CMakeFiles/brotlienc.dir/c/enc/memory.c.o
[ 89%] Building C object CMakeFiles/brotlienc.dir/c/enc/metablock.c.o
[ 93%] Building C object CMakeFiles/brotlienc.dir/c/enc/static_dict.c.o
[ 96%] Building C object CMakeFiles/brotlienc.dir/c/enc/utf8_util.c.o
[100%] Linking C static library libbrotlienc.a
[100%] Built target brotlienc

compile nginx with ngx_brotli pass.

I know this is an upstream question, but these projects belong to Google. So I won't close this issue

[FireMasterK]

There's an upstream issue at: https://github.com/google/brotli/issues/1094

[wyattoday]

@FireMasterK

Yes, as I wrote in the readme. Better to link the actual source than to copy & paste it in a random issue or forum post. "Why?" you ask? So the canonical source can be referenced and updated when things inevitable change in the future.

This is a compiler issue. Don't use ancient compilers. If you're on Redhat (or a clone) use the devtoolset / gcc-toolset to use the latest compilers (while still targeting the system glibc).

In short, google it.

[kn007]

It seems that it will not be fixed.

[saytesnake]

It seems that it will not be fixed.

Maybe it doesn't need fixing because it's really a compiler issue.

alpine:3.14 with gcc 10.3.1, compile pass without any errors, musl of course.

But Redhat 8 (or Rocky8 or AlmaLinux8) with gcc 8.5.0 or gcc 10.3.1 is compile failed, and I don't know why because I know nothing about C. 😅

Using gcc 11.2.1 will compile successfully.

dnf -y install gcc-toolset-11-gcc gcc-toolset-11-gcc-c++ source /opt/rh/gcc-toolset-11/enable

[kn007]

temp fix: change CMAKE_AR & CMAKE_RANLIB value from ngx_brotli/deps/brotli/out/CMakeFiles/3.27.7/CMakeCCompiler.cmake ar to gcc-ar, ranlib to gcc-ranlib

@saytesnake