FTBFS due to mismatch during inlining

[grawlinson]

Describe the bug FTBFS due to inlining.

To Reproduce Steps to reproduce the behavior:

1. Build package from AUR. EDIT: With the commit changed to point at 0.14.0.
2. See below CMake output:

[297/398] Building CXX object src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o
FAILED: src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o
/usr/bin/g++ -DBOOST_ASIO_SEPARATE_COMPILATION -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_BEAST_SEPARATE_COMPILATION -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FIBER_DYN_LINK -DBOOST_FIBER_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DUSE_ZMALLOC_MI -I/build/dragonflydb/src/dragonflydb -I/build/dragonflydb/src/dragonflydb/genfiles -I/build/dragonflydb/src/dragonflydb/src -I/build/dragonflydb/src/dragonflydb/helio -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/glog-build -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/glog-src/src -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/abseil_cpp-src -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/xxhash/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/mimalloc/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/lua/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/jsoncons/include -Wall -Wextra -g -fPIC -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -fno-omit-frame-pointer -Wno-unused-parameter -march=sandybridge -mtune=skylake -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -flto=auto -std=c++20 -DHAS_RAWMEMCHR -fdiagnostics-color=always  -O3 -DNDEBUG  -std=gnu++17 -MD -MT src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o -MF src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o.d -o src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o -c /build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc
In file included from /build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:13:
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h: In function ‘dfly::detail::simd_variant1_pack(char const*, char const*, unsigned char*)’:
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long __vector(2), long long __vector(2))’: target specific option mismatch
  136 | _mm_shuffle_epi8 (__m128i __X, __m128i __Y)
      | ^~~~~~~~~~~~~~~~
/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here
   65 |     val = _mm_shuffle_epi8(val, control);
      |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long __vector(2), long long __vector(2))’: target specific option mismatch
  136 | _mm_shuffle_epi8 (__m128i __X, __m128i __Y)
      | ^~~~~~~~~~~~~~~~
/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here
   65 |     val = _mm_shuffle_epi8(val, control);
      |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long __vector(2), long long __vector(2))’: target specific option mismatch
  136 | _mm_shuffle_epi8 (__m128i __X, __m128i __Y)
      | ^~~~~~~~~~~~~~~~
/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here
   65 |     val = _mm_shuffle_epi8(val, control);
      |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
[298/398] Building CXX object src/core/CMakeFiles/dfly_core.dir/tx_queue.cc.o

Environment (please complete the following information):

• OS: Arch Linux
• Kernel: 6.1.8
• Containerized?: Bare metal building system package in a systemd-nspawn container (has worked for the past few releases, and the error removes this as a cause).
• Dragonfly Version: 0.14.0

[romange]

@grawlinson what happens if you comment out #pragma GCC optimize("Ofast") at line 25 ?

[grawlinson]

It looks like it still fails when that line is commented out:

[301/398] Building CXX object src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o
FAILED: src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o
/usr/bin/g++ -DBOOST_ASIO_SEPARATE_COMPILATION -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_BEAST_SEPARATE_COMPILATION -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FIBER_DYN_LINK -DBOOST_FIBER_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DUSE_ZMALLOC_MI -I/build/dragonflydb/src/dragonflydb -I/build/dragonflydb/src/dragonflydb/genfiles -I/build/dragonflydb/src/dragonflydb/src -I/build/dragonflydb/src/dragonflydb/helio -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/glog-build -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/glog-src/src -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/abseil_cpp-src -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/xxhash/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/mimalloc/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/lua/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/jsoncons/include -Wall -Wextra -g -fPIC -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -fno-omit-frame-pointer -Wno-unused-parameter -march=sandybridge -mtune=skylake -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -g -ffile-prefix-map=/build/dragonflydb/src=/usr/src/debug/dragonflydb -flto=auto -std=c++20 -DHAS_RAWMEMCHR -fdiagnostics-color=always  -O3 -DNDEBUG  -std=gnu++17 -MD -MT src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o -MF src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o.d -o src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o -c /build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc
In file included from /build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:13:
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h: In function ‘dfly::detail::simd_variant1_pack(char const*, char const*, unsigned char*)’:
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long __vector(2), long long __vector(2))’: target specific option mismatch
  136 | _mm_shuffle_epi8 (__m128i __X, __m128i __Y)
      | ^~~~~~~~~~~~~~~~
/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here
   65 |     val = _mm_shuffle_epi8(val, control);
      |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long __vector(2), long long __vector(2))’: target specific option mismatch
  136 | _mm_shuffle_epi8 (__m128i __X, __m128i __Y)
      | ^~~~~~~~~~~~~~~~
/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here
   65 |     val = _mm_shuffle_epi8(val, control);
      |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long __vector(2), long long __vector(2))’: target specific option mismatch
  136 | _mm_shuffle_epi8 (__m128i __X, __m128i __Y)
      | ^~~~~~~~~~~~~~~~
/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here
   65 |     val = _mm_shuffle_epi8(val, control);
      |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
[302/398] Building CXX object src/core/CMakeFiles/dfly_core.dir/mi_memory_resource.cc.o

[romange]

I see that in the compile command you used these overrides " -march=sandybridge -mtune=skylake -march=x86-64 -mtune=generic" - you override the arch with x86-64 and it probably does not have the required instruction set. Try adding "-mavx" or restoring the original sandy bridge architecture.

On Mon, Jan 30, 2023 at 9:31 AM George Rawlinson @.***> wrote:

It looks like it still fails when that line is commented out:

[301/398] Building CXX object src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o

FAILED: src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o

/usr/bin/g++ -DBOOST_ASIO_SEPARATE_COMPILATION -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_BEAST_SEPARATE_COMPILATION -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FIBER_DYN_LINK -DBOOST_FIBER_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DUSE_ZMALLOC_MI -I/build/dragonflydb/src/dragonflydb -I/build/dragonflydb/src/dragonflydb/genfiles -I/build/dragonflydb/src/dragonflydb/src -I/build/dragonflydb/src/dragonflydb/helio -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/glog-build -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/glog-src/src -I/build/dragonflydb/src/dragonflydb/build-opt/_deps/abseil_cpp-src -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/xxhash/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/mimalloc/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/lua/include -isystem /build/dragonflydb/src/dragonflydb/build-opt/third_party/libs/jsoncons/include -Wall -Wextra -g -fPIC -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -fno-omit-frame-pointer -Wno-unused-parameter -march=sandybridge -mtune=skylake -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -g -ffile-prefix-map=/build/dragonflydb/src=/usr/src/debug/dragonflydb -flto=auto -std=c++20 -DHAS_RAWMEMCHR -fdiagnostics-color=always -O3 -DNDEBUG -std=gnu++17 -MD -MT src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o -MF src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o.d -o src/core/CMakeFiles/dfly_core.dir/detail/bitpacking.cc.o -c /build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc

In file included from /build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:13:

/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h: In function ‘dfly::detail::simd_variant1_pack(char const, char const, unsigned char*)’:

/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long vector(2), long long vector(2))’: target specific option mismatch

136 | _mm_shuffle_epi8 (m128i X, m128i Y)

  | ^~~~~~~~~~~~~~~~

/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here

65 | val = _mm_shuffle_epi8(val, control);

  |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~

/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long vector(2), long long vector(2))’: target specific option mismatch

136 | _mm_shuffle_epi8 (m128i X, m128i Y)

  | ^~~~~~~~~~~~~~~~

/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here

65 | val = _mm_shuffle_epi8(val, control);

  |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~

/usr/lib/gcc/x86_64-pc-linux-gnu/12.2.1/include/tmmintrin.h:136:1: error: inlining failed in call to ‘always_inline’ ‘_mm_shuffle_epi8(long long vector(2), long long vector(2))’: target specific option mismatch

136 | _mm_shuffle_epi8 (m128i X, m128i Y)

  | ^~~~~~~~~~~~~~~~

/build/dragonflydb/src/dragonflydb/src/core/detail/bitpacking.cc:65:27: note: called from here

65 | val = _mm_shuffle_epi8(val, control);

  |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~

[302/398] Building CXX object src/core/CMakeFiles/dfly_core.dir/mi_memory_resource.cc.o

— Reply to this email directly, view it on GitHub https://github.com/dragonflydb/dragonfly/issues/732#issuecomment-1408118933, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4BFCE7RP4RR7HGQFK5XNLWU5U37ANCNFSM6AAAAAAUKQOK5U . You are receiving this because you commented.Message ID: @.***>

[grawlinson]

We set these flags in /etc/makepkg.conf as shown below since Arch Linux currently only supports the x86_64 architecture.

#-- Compiler and Linker Flags
#CPPFLAGS=""
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions \
        -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security \
        -fstack-clash-protection -fcf-protection"
CXXFLAGS="$CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"

Is there a way to force dragonfly to respect these particular flags rather than adding skylake + sandylake?

[romange]

@Pothulapati would you be able to help me with this? I would like to extend our CI to run in a special build mode that builds dragonfly with different flags. if you have spare cycles, I can talk to you offline but basically, I would like to have a daily scheduled run with separate set of flags that tests dragonfly for non-standard architectures.

[romange]

This branch https://github.com/dragonflydb/dragonfly/tree/AdaptGeneric has potential fix to this problem but I need to see it works on aarch64 as well.

[Pothulapati]

I and @romange discussed offline on this. I will be raising a PR that runs every day with common build flags so that we can verify that building from the source is possible, for common configurations with known flags.

[romange]

Right, at the current codebase state, your build should fail with these flags. It's a good thing, meaning you reproduce a problem. The fix is in the branch above but I want to see the pipeline failing first.