search

mhx / dwarfs

A fast high compression read-only file system for Linux, Windows and macOS
GNU General Public License v3.0
2.16k stars 58 forks source link

Issues when building into different architectures and on musl #185

Closed NafzorOB closed 10 months ago

NafzorOB commented 10 months ago

When trying to cross-compile dwarfs for aarch64 and armv7l the build fails with the following output:

-- Performing Test FOLLY_HAVE_IFUNC
-- Performing Test FOLLY_HAVE_IFUNC - Success
-- Performing Test FOLLY_HAVE_STD__IS_TRIVIALLY_COPYABLE
-- Performing Test FOLLY_HAVE_STD__IS_TRIVIALLY_COPYABLE - Success
-- Performing Test FOLLY_HAVE_UNALIGNED_ACCESS
CMake Error: try_run() invoked in cross-compiling mode, please set the following cache variables appropriately:
   FOLLY_HAVE_UNALIGNED_ACCESS_EXITCODE (advanced)
For details see /builddir/dwarfs-0.7.5/build/TryRunResults.cmake
-- Performing Test FOLLY_HAVE_UNALIGNED_ACCESS - Failed
-- Performing Test FOLLY_HAVE_VLA
-- Performing Test FOLLY_HAVE_VLA - Success
-- Performing Test FOLLY_HAVE_WEAK_SYMBOLS
CMake Error: try_run() invoked in cross-compiling mode, please set the following cache variables appropriately:
   FOLLY_HAVE_WEAK_SYMBOLS_EXITCODE (advanced)
For details see /builddir/dwarfs-0.7.5/build/TryRunResults.cmake
-- Performing Test FOLLY_HAVE_WEAK_SYMBOLS - Failed
-- Performing Test FOLLY_HAVE_LINUX_VDSO
CMake Error: try_run() invoked in cross-compiling mode, please set the following cache variables appropriately:
   FOLLY_HAVE_LINUX_VDSO_EXITCODE (advanced)
For details see /builddir/dwarfs-0.7.5/build/TryRunResults.cmake
-- Performing Test FOLLY_HAVE_LINUX_VDSO - Failed
-- Performing Test FOLLY_HAVE_WCHAR_SUPPORT
CMake Error: try_run() invoked in cross-compiling mode, please set the following cache variables appropriately:
   FOLLY_HAVE_WCHAR_SUPPORT_EXITCODE (advanced)
For details see /builddir/dwarfs-0.7.5/build/TryRunResults.cmake
-- Performing Test FOLLY_HAVE_WCHAR_SUPPORT - Failed
-- Performing Test FOLLY_HAVE_EXTRANDOM_SFMT19937
-- Performing Test FOLLY_HAVE_EXTRANDOM_SFMT19937 - Success
-- Performing Test HAVE_VSNPRINTF_ERRORS
CMake Error: try_run() invoked in cross-compiling mode, please set the following cache variables appropriately:
   HAVE_VSNPRINTF_ERRORS_EXITCODE (advanced)
For details see /builddir/dwarfs-0.7.5/build/TryRunResults.cmake
-- Performing Test HAVE_VSNPRINTF_ERRORS - Failed
-- Performing Test GFLAGS_NAMESPACE_IS_GFLAGS
-- Performing Test GFLAGS_NAMESPACE_IS_GFLAGS - Success
-- Performing Test COMPILER_HAS_F_COROUTINES
-- Performing Test COMPILER_HAS_F_COROUTINES - Success
-- GCC has support for C++ coroutines, setting flag for Folly build.
-- arch  does not match x86_64, skipping building SSE4.2 version of base64
-- Performing Test COMPILER_HAS_M_PCLMUL
-- Performing Test COMPILER_HAS_M_PCLMUL - Failed
-- compiler does not have flag pclmul, skipping setting compile flags for /builddir/dwarfs-0.7.5/folly/folly/hash/detail/ChecksumDetail.cpp;/builddir/dwarfs-0.7.5/folly/folly/hash/detail/Crc32CombineDetail.cpp;/builddir/dwarfs-0.7.5/folly/folly/hash/detail/Crc32cDetail.cpp
CMake Deprecation Warning at fbthrift/CMakeLists.txt:15 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- Found Boost: /usr/aarch64-linux-gnu/usr/lib64/cmake/Boost-1.83.0/BoostConfig.cmake (found version "1.83.0") found components: filesystem 
-- Found OpenSSL: /usr/aarch64-linux-gnu/usr/lib/libcrypto.so (found version "3.1.4")  
CMake Deprecation Warning at zstd/build/cmake/CMakeLists.txt:10 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- ZSTD VERSION: 1.5.5
-- Performing Test C_FLAG_WALL
-- Performing Test C_FLAG_WALL - Success
-- Performing Test CXX_FLAG_WALL
-- Performing Test CXX_FLAG_WALL - Success
-- Performing Test C_FLAG_WEXTRA
-- Performing Test C_FLAG_WEXTRA - Success
-- Performing Test CXX_FLAG_WEXTRA
-- Performing Test CXX_FLAG_WEXTRA - Success
-- Performing Test C_FLAG_WUNDEF
-- Performing Test C_FLAG_WUNDEF - Success
-- Performing Test CXX_FLAG_WUNDEF
-- Performing Test CXX_FLAG_WUNDEF - Success
-- Performing Test C_FLAG_WSHADOW
-- Performing Test C_FLAG_WSHADOW - Success
-- Performing Test CXX_FLAG_WSHADOW
-- Performing Test CXX_FLAG_WSHADOW - Success
-- Performing Test C_FLAG_WCAST_ALIGN
-- Performing Test C_FLAG_WCAST_ALIGN - Success
-- Performing Test CXX_FLAG_WCAST_ALIGN
-- Performing Test CXX_FLAG_WCAST_ALIGN - Success
-- Performing Test C_FLAG_WCAST_QUAL
-- Performing Test C_FLAG_WCAST_QUAL - Success
-- Performing Test CXX_FLAG_WCAST_QUAL
-- Performing Test CXX_FLAG_WCAST_QUAL - Success
-- Performing Test C_FLAG_WSTRICT_PROTOTYPES
-- Performing Test C_FLAG_WSTRICT_PROTOTYPES - Success
-- Performing Test LD_FLAG_Z_NOEXECSTACK
-- Performing Test LD_FLAG_Z_NOEXECSTACK - Success
-- Performing Test C_FLAG_QUNUSED_ARGUMENTS
-- Performing Test C_FLAG_QUNUSED_ARGUMENTS - Failed
-- Performing Test CXX_FLAG_QUNUSED_ARGUMENTS
-- Performing Test CXX_FLAG_QUNUSED_ARGUMENTS - Failed
-- Performing Test C_FLAG_WA_NOEXECSTACK
-- Performing Test C_FLAG_WA_NOEXECSTACK - Success
-- Performing Test CXX_FLAG_WA_NOEXECSTACK
-- Performing Test CXX_FLAG_WA_NOEXECSTACK - Success
-- CMAKE_INSTALL_PREFIX: /usr
-- CMAKE_INSTALL_LIBDIR: lib64
-- ZSTD_LEGACY_SUPPORT defined!
-- ZSTD_MULTITHREAD_SUPPORT is enabled
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /builddir/dwarfs-0.7.5/build/googletest-download
[1/9] Creating directories for 'googletest'
[2/9] Performing download step (git clone) for 'googletest'
Cloning into 'googletest-src'...
Already on 'main'
Your branch is up to date with 'origin/main'.
[3/9] Performing update step for 'googletest'
[4/9] No patch step for 'googletest'
[5/9] No configure step for 'googletest'
[6/9] No build step for 'googletest'
[7/9] No install step for 'googletest'
[8/9] No test step for 'googletest'
[9/9] Completed 'googletest'
-- Configuring incomplete, errors occurred!

A different error also happens when cross-building using musl instead of glibc:

[138/547] Building CXX object folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/DwarfLineNumberVM.cpp.o
[139/547] Building CXX object folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/DwarfSection.cpp.o
[140/547] Building CXX object folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/Elf.cpp.o
FAILED: folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/Elf.cpp.o 
/usr/bin/ccache /usr/bin/g++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFOLLY_CFG_NO_COROUTINES -DFOLLY_XLOG_STRIP_PREFIXES=\"/builddir/dwarfs-0.7.5:/builddir/dwarfs-0.7.5/build\" -DGFLAGS_IS_A_DLL=0 -DGLOG_NO_ABBREVIATED_SEVERITIES -DNOGDI -DNOMINMAX -D_GNU_SOURCE -D_REENTRANT -I/builddir/dwarfs-0.7.5/build/fmtlib-install/include -I/builddir/dwarfs-0.7.5/folly -I/builddir/dwarfs-0.7.5/build/folly -DNDEBUG -fstack-clash-protection -D_FORTIFY_SOURCE=2 -mtune=generic -O2    -ffile-prefix-map=/builddir/dwarfs-0.7.5/build=. -std=gnu++20 -fdiagnostics-color=always -fcoroutines -g -finput-charset=UTF-8 -fsigned-char -Wall -Wno-deprecated -Wno-deprecated-declarations -Wno-sign-compare -Wno-unused -Wuninitialized -Wunused-label -Wunused-result -Wshadow-compatible-local -Wno-noexcept-type -faligned-new -fopenmp -MD -MT folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/Elf.cpp.o -MF folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/Elf.cpp.o.d -o folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/Elf.cpp.o -c /builddir/dwarfs-0.7.5/folly/folly/experimental/symbolizer/Elf.cpp
../folly/folly/experimental/symbolizer/Elf.cpp: In member function 'folly::symbolizer::ElfFile::OpenResult folly::symbolizer::ElfFile::init()':
../folly/folly/experimental/symbolizer/Elf.cpp:246:27: error: 'ELFCLASSFOLLY_ELF_NATIVE_CLASS' was not declared in this scope
  246 | #define EXPECTED_CLASS P1(ELFCLASS, FOLLY_ELF_NATIVE_CLASS)
      |                           ^~~~~~~~
../folly/folly/experimental/symbolizer/Elf.cpp:248:18: note: in definition of macro 'P2'
  248 | #define P2(a, b) a##b
      |                  ^
../folly/folly/experimental/symbolizer/Elf.cpp:246:24: note: in expansion of macro 'P1'
  246 | #define EXPECTED_CLASS P1(ELFCLASS, FOLLY_ELF_NATIVE_CLASS)
      |                        ^~
../folly/folly/experimental/symbolizer/Elf.cpp:250:38: note: in expansion of macro 'EXPECTED_CLASS'
  250 |   if (elfHeader.e_ident[EI_CLASS] != EXPECTED_CLASS) {
      |                                      ^~~~~~~~~~~~~~
[141/547] Building CXX object folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/DwarfUtil.cpp.o
[142/547] Building CXX object folly/CMakeFiles/folly_base.dir/folly/experimental/observer/detail/ObserverManager.cpp.o
In file included from ../folly/folly/experimental/observer/detail/Core.h:19,
                 from ../folly/folly/experimental/observer/detail/ObserverManager.h:22,
                 from ../folly/folly/experimental/observer/detail/ObserverManager.cpp:17:
In member function 'std::size_t folly::Function<FunctionType>::exec(Op, Data*, Data*) const [with FunctionType = std::shared_ptr<folly::observer_detail::Core>()]',
    inlined from 'folly::Function<FunctionType>::~Function() [with FunctionType = std::shared_ptr<folly::observer_detail::Core>()]' at ../folly/folly/Function.h:808:21,
    inlined from 'void folly::Optional<Value>::StorageNonTriviallyDestructible::clear() [with Value = folly::Function<std::shared_ptr<folly::observer_detail::Core>()>]' at ../folly/folly/Optional.h:462:21,
    inlined from 'folly::Optional<Value>::StorageNonTriviallyDestructible::~StorageNonTriviallyDestructible() [with Value = folly::Function<std::shared_ptr<folly::observer_detail::Core>()>]' at ../folly/folly/Optional.h:457:47,
    inlined from 'folly::Optional<folly::Function<std::shared_ptr<folly::observer_detail::Core>()> >::~Optional()' at ../folly/folly/Optional.h:107:7,
    inlined from 'folly::Optional<T> folly::UnboundedQueue<T, SingleProducer, SingleConsumer, MayBlock, LgSegmentSize, LgAlign, Atom>::tryDequeueUntilSC(Segment*, const std::chrono::time_point<_Clock, _Dur>&) [with Clock = std::chrono::_V2::steady_clock; Duration = std::chrono::duration<long int, std::ratio<1, 1000000000> >; T = folly::Function<std::shared_ptr<folly::observer_detail::Core>()>; bool SingleProducer = false; bool SingleConsumer = true; bool MayBlock = true; long unsigned int LgSegmentSize = 8; long unsigned int LgAlign = 7; Atom = std::atomic]' at ../folly/folly/concurrency/UnboundedQueue.h:458:3,
    inlined from 'folly::Optional<T> folly::UnboundedQueue<T, SingleProducer, SingleConsumer, MayBlock, LgSegmentSize, LgAlign, Atom>::tryDequeueUntil(const std::chrono::time_point<_Clock, _Dur>&) [with Clock = std::chrono::_V2::steady_clock; Duration = std::chrono::duration<long int, std::ratio<1, 1000000000> >; T = folly::Function<std::shared_ptr<folly::observer_detail::Core>()>; bool SingleProducer = false; bool SingleConsumer = true; bool MayBlock = true; long unsigned int LgSegmentSize = 8; long unsigned int LgAlign = 7; Atom = std::atomic]' at ../folly/folly/concurrency/UnboundedQueue.h:429:43,
    inlined from 'folly::Optional<T> folly::UnboundedQueue<T, SingleProducer, SingleConsumer, MayBlock, LgSegmentSize, LgAlign, Atom>::try_dequeue() [with T = folly::Function<std::shared_ptr<folly::observer_detail::Core>()>; bool SingleProducer = false; bool SingleConsumer = true; bool MayBlock = true; long unsigned int LgSegmentSize = 8; long unsigned int LgAlign = 7; Atom = std::atomic]' at ../folly/folly/concurrency/UnboundedQueue.h:289:72,
    inlined from 'bool folly::UnboundedQueue<T, SingleProducer, SingleConsumer, MayBlock, LgSegmentSize, LgAlign, Atom>::try_dequeue(T&) [with T = folly::Function<std::shared_ptr<folly::observer_detail::Core>()>; bool SingleProducer = false; bool SingleConsumer = true; bool MayBlock = true; long unsigned int LgSegmentSize = 8; long unsigned int LgAlign = 7; Atom = std::atomic]' at ../folly/folly/concurrency/UnboundedQueue.h:280:10,
    inlined from 'folly::observer_detail::ObserverManager::UpdatesManager::NextQueueProcessor::NextQueueProcessor()::<lambda()>' at ../folly/folly/experimental/observer/detail/ObserverManager.cpp:125:36:
../folly/folly/Function.h:656:10: warning: 'ret.folly::Optional<folly::Function<std::shared_ptr<folly::observer_detail::Core>()> >::storage_.folly::Optional<folly::Function<std::shared_ptr<folly::observer_detail::Core>()> >::StorageNonTriviallyDestructible::<unnamed>.folly::Optional<folly::Function<std::shared_ptr<folly::observer_detail::Core>()> >::StorageNonTriviallyDestructible::._anon_135::value.folly::Function<std::shared_ptr<folly::observer_detail::Core>()>::exec_' may be used uninitialized [-Wmaybe-uninitialized]
  656 |     if (!exec_) {
      |          ^~~~~
In file included from ../folly/folly/executors/TimedDrivableExecutor.h:21,
                 from ../folly/folly/futures/Future.h:34,
                 from ../folly/folly/experimental/observer/detail/Core.h:21:
../folly/folly/concurrency/UnboundedQueue.h: In lambda function:
../folly/folly/concurrency/UnboundedQueue.h:453:24: note: 'ret' declared here
  453 |     folly::Optional<T> ret = e.takeItem();
      |                        ^~~
[143/547] Building CXX object folly/CMakeFiles/folly_base.dir/folly/experimental/symbolizer/ElfCache.cpp.o
ninja: build stopped: subcommand failed.
mhx commented 10 months ago

Hi and thanks for your report.

I'm not aware that anyone has tried cross-compiling this project yet. It does build fine natively on aarch64, though.

As for armv7l, you're out of luck, as 32-bit platforms are not supported — I'll update the documentation to make this clear.

I'd be more than happy to accept PRs that fix/enable cross-compilation.

Maybe you can leave some more details on this issue regarding the environment and options you used for cross-compilation.

NafzorOB commented 10 months ago

Thanks for your response. After some investigation seems like Folly doesnt support cross-compiling on cmake. This isnt an issue with dwarfs but with Folly itself, so i will close this issue.

mhx commented 10 months ago

Thanks for digging out that thread, there's quite a bit of information there. Cross-compilation would definitely be interesting in the future.