search

facebookincubator / velox

A composable and fully extensible C++ execution engine library for data management systems.
https://velox-lib.io/
Apache License 2.0
3.51k stars 1.15k forks source link

M1 build failing with C++ compilation errors #8331

Open s0nskar opened 10 months ago

s0nskar commented 10 months ago

Problem description

Getting below compilation error while building velox in Mac M1, does anyone have any pointers.

Steps i ran:

$ CPU_TARGET="arm64" ./scripts/setup-macos.sh
$ CPU_TARGET="arm64" make

Error:

FAILED: velox/common/base/CMakeFiles/velox_common_base.dir/RuntimeMetrics.cpp.o 
/opt/homebrew/bin/ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_SHARED -DFOLLY_HAVE_INT128_T=1 -DGFLAGS_IS_A_DLL=0 -I/Users/sanskarm/OSS/velox/_build/release/_deps/protobuf-src/src -I/Users/sanskarm/OSS/velox/. -I/Users/sanskarm/OSS/velox/velox/external/xxhash -I/Users/sanskarm/OSS/velox/_build/release/_deps/xsimd-src/include -isystem /Users/sanskarm/OSS/velox/velox -isystem /Users/sanskarm/OSS/velox/velox/external -isystem /opt/homebrew/include -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness -Werror -O3 -DNDEBUG -std=gnu++17 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk -fPIC -fcolor-diagnostics -MD -MT velox/common/base/CMakeFiles/velox_common_base.dir/RuntimeMetrics.cpp.o -MF velox/common/base/CMakeFiles/velox_common_base.dir/RuntimeMetrics.cpp.o.d -o velox/common/base/CMakeFiles/velox_common_base.dir/RuntimeMetrics.cpp.o -c /Users/sanskarm/OSS/velox/velox/common/base/RuntimeMetrics.cpp
In file included from /Users/sanskarm/OSS/velox/velox/common/base/RuntimeMetrics.cpp:17:
In file included from /opt/homebrew/include/folly/ThreadLocal.h:52:
In file included from /opt/homebrew/include/folly/detail/ThreadLocalDetail.h:30:
In file included from /opt/homebrew/include/folly/Exception.h:25:
In file included from /opt/homebrew/include/folly/Conv.h:128:
In file included from /opt/homebrew/include/folly/Demangle.h:19:
In file included from /opt/homebrew/include/folly/FBString.h:34:
In file included from /opt/homebrew/include/fmt/format.h:49:
/opt/homebrew/include/fmt/core.h:1600:63: error: implicit instantiation of undefined template 'fmt::detail::type_is_unformattable_for<const facebook::velox::RuntimeCounter::Unit, char>'
    type_is_unformattable_for<T, typename Context::char_type> _;
                                                              ^
/opt/homebrew/include/fmt/core.h:1842:23: note: in instantiation of function template specialization 'fmt::detail::make_arg<true, fmt::basic_format_context<fmt::appender, char>, const facebook::velox::RuntimeCounter::Unit, 0>' requested here
        data_{detail::make_arg<is_packed, Context>(args)...} {
                      ^
/opt/homebrew/include/fmt/core.h:1860:10: note: in instantiation of function template specialization 'fmt::format_arg_store<fmt::basic_format_context<fmt::appender, char>, facebook::velox::RuntimeCounter::Unit, facebook::velox::RuntimeCounter::Unit>::format_arg_store<const facebook::velox::RuntimeCounter::Unit, const facebook::velox::RuntimeCounter::Unit>' requested here
  return {args...};
         ^
/Users/sanskarm/OSS/velox/./velox/common/base/Exceptions.h:153:33: note: in instantiation of function template specialization 'fmt::make_format_args<fmt::basic_format_context<fmt::appender, char>, const facebook::velox::RuntimeCounter::Unit, const facebook::velox::RuntimeCounter::Unit>' requested here
  return fmt::vformat(fmt, fmt::make_format_args(args...));
                                ^
/Users/sanskarm/OSS/velox/velox/common/base/RuntimeMetrics.cpp:38:3: note: in instantiation of function template specialization 'facebook::velox::detail::errorMessage<facebook::velox::RuntimeCounter::Unit, facebook::velox::RuntimeCounter::Unit>' requested here
  VELOX_CHECK_EQ(unit, other.unit);
  ^
/Users/sanskarm/OSS/velox/./velox/common/base/Exceptions.h:253:37: note: expanded from macro 'VELOX_CHECK_EQ'
#define VELOX_CHECK_EQ(e1, e2, ...) _VELOX_CHECK_OP(e1, e2, ==, ##__VA_ARGS__)
                                    ^
/Users/sanskarm/OSS/velox/./velox/common/base/Exceptions.h:230:26: note: expanded from macro '_VELOX_CHECK_OP'
  _VELOX_CHECK_OP_HELPER(_VELOX_CHECK_IMPL, expr1, expr2, op, ##__VA_ARGS__)
                         ^
/opt/homebrew/include/fmt/core.h:1578:45: note: template is declared here
template <typename T, typename Char> struct type_is_unformattable_for;
                                            ^
/opt/homebrew/include/fmt/core.h:1603:3: error: static assertion failed due to requirement 'formattable': Cannot format an argument. To make type T formattable provide a formatter<T> specialization: https://fmt.dev/latest/api.html#udt
  static_assert(
  ^
2 errors generated.
[182/1439] Building CXX object velox/common/base/tests/CMakeFiles/velox_base_test.dir/ExceptionTest.cpp.o
ninja: build stopped: subcommand failed.
make[1]: *** [build] Error 1
make: *** [release] Error 2

System information

Velox System Info v0.0.2
Commit: e96f0d5ac2aa0fe45e0fc27319a62cffe046a8b7
CMake Version: 3.28.1
System: Darwin-23.2.0
Arch: arm64
C++ Compiler: /Library/Developer/CommandLineTools/usr/bin/c++
C++ Compiler Version: 15.0.0.15000100
C Compiler: /Library/Developer/CommandLineTools/usr/bin/cc
C Compiler Version: 15.0.0.15000100
CMake Prefix Path: /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/usr;/opt/homebrew;/usr/local;/usr;/;/opt/homebrew/Cellar/cmake/3.28.1;/usr/local;/usr/X11R6;/usr/pkg;/opt;/sw;/opt/local

CMake log

No response

pedroerp commented 10 months ago

@assignUser @kgpai could this be related to the recent libfmt upgrade?

kgpai commented 10 months ago

@pedroerp I dont believe we have updated fmt yet , https://github.com/facebookincubator/velox/pull/7941 (this is still open). Is there some other else I am not aware of ?

@s0nskar I strongly suspect that here you are linking against a different version of fmt installed in your system. Can you share your cmake logs (the ones immediately following make debug before compilation starts) ?

Another thing to do is run the following fmt_SOURCE=BUNDLED make debug

s0nskar commented 10 months ago

@kgpai Added the info you requested.

Can you share your cmake logs (the ones immediately following make debug before compilation starts) ?

CPU_TARGET="arm64" make debug                   
/Library/Developer/CommandLineTools/usr/bin/make cmake BUILD_DIR=debug BUILD_TYPE=Debug
mkdir -p _build/debug && \
    cmake -B \
        "_build/debug" \
        -DTREAT_WARNINGS_AS_ERRORS=1 -DENABLE_ALL_WARNINGS=1 -DVELOX_BUILD_MINIMAL="OFF" -DVELOX_BUILD_TESTING="ON" -DCMAKE_BUILD_TYPE=Debug \
        -GNinja -DMAX_LINK_JOBS= -DMAX_HIGH_MEM_JOBS= -DVELOX_FORCE_COLORED_OUTPUT=ON \

-- Build type: Debug
-- Building cpr from source
-- CXX standard: 17
-- C++ Requests CMake Options
-- =======================================================
--   CPR_GENERATE_COVERAGE: OFF
--   CPR_CURL_NOSIGNAL: OFF
--   CURL_VERBOSE_LOGGING: OFF
--   CPR_USE_SYSTEM_GTEST: OFF
--   CPR_USE_SYSTEM_CURL: OFF
--   CPR_ENABLE_CURL_HTTP_ONLY: ON
--   CPR_ENABLE_SSL: ON
--   CPR_FORCE_OPENSSL_BACKEND: OFF
--   CPR_FORCE_WINSSL_BACKEND: OFF
--   CPR_FORCE_DARWINSSL_BACKEND: OFF
--   CPR_FORCE_MBEDTLS_BACKEND: OFF
--   CPR_ENABLE_LINTING: OFF
--   CPR_ENABLE_CPPCHECK: OFF
--   CPR_BUILD_TESTS: OFF
--   CPR_BUILD_TESTS_SSL: OFF
--   CPR_BUILD_TESTS_PROXY: OFF
--   CPR_SKIP_CA_BUNDLE_SEARCH: OFF
--   CPR_USE_BOOST_FILESYSTEM: OFF
--   CPR_DEBUG_SANITIZER_FLAG_THREAD: OFF
--   CPR_DEBUG_SANITIZER_FLAG_ADDR: OFF
--   CPR_DEBUG_SANITIZER_FLAG_LEAK: OFF
--   CPR_DEBUG_SANITIZER_FLAG_UB: OFF
--   CPR_DEBUG_SANITIZER_FLAG_ALL: OFF
-- =======================================================
-- Automatically detecting SSL backend.
-- Detecting SSL backend...
-- SSL auto detect: Using DarwinSSL.
-- Configuring built-in curl...
-- Enabled curl SSL
-- curl version=[8.4.0]
-- Picky compiler options: -Wextra -Wall -pedantic -Wbad-function-cast -Wconversion -Winline -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wno-long-long -Wno-multichar -Wpointer-arith -Wshadow -Wsign-compare -Wundef -Wunused -Wwrite-strings -Wcast-align -Wdeclaration-after-statement -Wempty-body -Wendif-labels -Wfloat-equal -Wignored-qualifiers -Wno-format-nonliteral -Wno-sign-conversion -Wno-system-headers -Wstrict-prototypes -Wtype-limits -Wvla -Wshift-sign-overflow -Wshorten-64-to-32 -Wdouble-promotion -Wenum-conversion -Wunused-const-variable -Wcomma -Wmissing-variable-declarations -Wassign-enum -Wextra-semi-stmt
-- Could NOT find LibPSL (missing: LIBPSL_LIBRARY LIBPSL_INCLUDE_DIR) 
-- CA path only supported by OpenSSL, wolfSSL, GnuTLS or mbedTLS. Set CURL_CA_PATH=none or enable one of those TLS backends.
-- Enabled features: SSL IPv6 unixsockets AsynchDNS Largefile alt-svc HSTS HTTPS-proxy threadsafe
-- Enabled protocols: HTTP HTTPS SCP SFTP
-- Enabled SSL backends: Secure Transport
-- Using ccache: /opt/homebrew/bin/ccache
Setting CMAKE_CXX_FLAGS=-mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden
FINAL CMAKE_CXX_FLAGS= -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness
-- Setting Boost source to AUTO
-- Found Boost: /opt/homebrew/lib/cmake/Boost-1.83.0/BoostConfig.cmake (found suitable version "1.83.0", minimum required is "1.66.0") found components: headers atomic context date_time filesystem program_options regex system thread 
-- Using SYSTEM Boost
-- Setting gflags source to AUTO
-- Using SYSTEM gflags
-- Setting DuckDB source to AUTO
CMake Warning at CMake/ResolveDependency.cmake:70 (find_package):
  By not providing "FindDuckDB.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "DuckDB", but
  CMake did not find one.

  Could not find a package configuration file provided by "DuckDB" with any
  of the following names:

    DuckDBConfig.cmake
    duckdb-config.cmake

  Add the installation prefix of "DuckDB" to CMAKE_PREFIX_PATH or set
  "DuckDB_DIR" to a directory containing one of the above files.  If "DuckDB"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  CMakeLists.txt:421 (resolve_dependency)

-- Building DuckDB from source
CMake Deprecation Warning at _build/debug/_deps/duckdb-src/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

CMake Warning at _build/debug/_deps/duckdb-src/CMakeLists.txt:162 (message):
  Not disabling vptr sanitizer on M1 Macbook - set DISABLE_VPTR_SANITIZER
  manually if you run into issues with false positives in the sanitizer

Treating warnings as errors.
CMake Deprecation Warning at _build/debug/_deps/duckdb-src/third_party/libpg_query/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

CMake Deprecation Warning at _build/debug/_deps/duckdb-src/third_party/re2/CMakeLists.txt:5 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- Using BUNDLED DuckDB
-- Setting fmt source to AUTO
-- Using SYSTEM fmt
-- Setting re2 source to AUTO
-- Using SYSTEM re2
-- Setting simdjson source to AUTO
CMake Warning at CMake/ResolveDependency.cmake:70 (find_package):
  By not providing "Findsimdjson.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "simdjson",
  but CMake did not find one.

  Could not find a package configuration file provided by "simdjson"
  (requested version 3.1.5) with any of the following names:

    simdjsonConfig.cmake
    simdjson-config.cmake

  Add the installation prefix of "simdjson" to CMAKE_PREFIX_PATH or set
  "simdjson_DIR" to a directory containing one of the above files.  If
  "simdjson" provides a separate development package or SDK, be sure it has
  been installed.
Call Stack (most recent call first):
  CMakeLists.txt:453 (resolve_dependency)

-- Building simdjson from source
-- Using BUNDLED simdjson
-- Setting folly source to AUTO
-- Found Boost: /opt/homebrew/lib/cmake/Boost-1.83.0/BoostConfig.cmake (found suitable version "1.83.0", minimum required is "1.51.0") found components: context filesystem program_options regex system thread 
-- Found folly: /opt/homebrew
-- Using SYSTEM folly
-- Setting Protobuf source to AUTO
-- Could NOT find Protobuf (missing: Protobuf_LIBRARIES Protobuf_INCLUDE_DIR) (Required is exact version "3.21")
-- Building Protobuf from source
-- 
-- 3.21.4.0
-- Using BUNDLED Protobuf
-- Found Bison keg installed by Homebrew at /opt/homebrew/opt/bison
-- Found Flex keg installed by Homebrew at /opt/homebrew/opt/flex
CMake Warning (dev) at /opt/homebrew/Cellar/cmake/3.28.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (GTest) does
  not match the name of the calling package (gtest).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  /opt/homebrew/Cellar/cmake/3.28.1/share/cmake/Modules/Findgtest.cmake:270 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  CMake/ResolveDependency.cmake:70 (find_package)
  CMakeLists.txt:537 (resolve_dependency)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY) 
-- Building gtest from source
-- Using BUNDLED gtest
CMake Warning (dev) at CMakeLists.txt:538 (set):
  Cannot set "VELOX_GTEST_INCUDE_DIR": current scope has no parent.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Building xsimd from source
CMake Deprecation Warning at _build/debug/_deps/xsimd-src/CMakeLists.txt:12 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- xsimd v10.0.0
CMake Deprecation Warning at velox/tpch/gen/dbgen/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- Configuring done (1.9s)
-- Generating done (0.8s)
-- Build files have been written to: /Users/sanskarm/OSS/velox/_build/debug

Another thing to do is run the following fmt_SOURCE=BUNDLED make debug

Tried running this command CPU_TARGET="arm64" fmt_SOURCE=BUNDLED make debug got another error

[678/1925] Building CXX object velox/common/base/tests/CMakeFiles/velox_base_test.dir/StatsReporterTest.cpp.o
FAILED: velox/common/base/tests/CMakeFiles/velox_base_test.dir/StatsReporterTest.cpp.o 
/opt/homebrew/bin/ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DFOLLY_HAVE_INT128_T=1 -DGFLAGS_IS_A_DLL=0 -I/Users/sanskarm/OSS/velox/_build/debug/_deps/protobuf-src/src -I/Users/sanskarm/OSS/velox/. -I/Users/sanskarm/OSS/velox/velox/external/xxhash -I/Users/sanskarm/OSS/velox/_build/debug/_deps/fmt-src/include -I/Users/sanskarm/OSS/velox/_build/debug/_deps/xsimd-src/include -isystem /Users/sanskarm/OSS/velox/velox -isystem /Users/sanskarm/OSS/velox/velox/external -isystem /opt/homebrew/include -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/googletest/include -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/googletest -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/googlemock/include -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/googlemock -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/include -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness -Werror -g -std=gnu++17 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk -fPIE -fcolor-diagnostics -MD -MT velox/common/base/tests/CMakeFiles/velox_base_test.dir/StatsReporterTest.cpp.o -MF velox/common/base/tests/CMakeFiles/velox_base_test.dir/StatsReporterTest.cpp.o.d -o velox/common/base/tests/CMakeFiles/velox_base_test.dir/StatsReporterTest.cpp.o -c /Users/sanskarm/OSS/velox/velox/common/base/tests/StatsReporterTest.cpp
/Users/sanskarm/OSS/velox/velox/common/base/tests/StatsReporterTest.cpp:137:10: error: 'init' is deprecated: Use the RAII version Init [-Werror,-Wdeprecated-declarations]
  folly::init(&argc, &argv, false);
         ^
/opt/homebrew/include/folly/init/Init.h:99:3: note: 'init' has been explicitly marked deprecated here
[[deprecated("Use the RAII version Init")]] void init(
  ^
1 error generated.
[684/1925] Building CXX object velox/common/base/tests/CMakeFiles/velox_memcpy_meter.dir/Memcpy.cpp.o
FAILED: velox/common/base/tests/CMakeFiles/velox_memcpy_meter.dir/Memcpy.cpp.o 
/opt/homebrew/bin/ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DFOLLY_HAVE_INT128_T=1 -DGFLAGS_IS_A_DLL=0 -I/Users/sanskarm/OSS/velox/_build/debug/_deps/protobuf-src/src -I/Users/sanskarm/OSS/velox/. -I/Users/sanskarm/OSS/velox/velox/external/xxhash -I/Users/sanskarm/OSS/velox/_build/debug/_deps/fmt-src/include -I/Users/sanskarm/OSS/velox/_build/debug/_deps/xsimd-src/include -isystem /Users/sanskarm/OSS/velox/velox -isystem /Users/sanskarm/OSS/velox/velox/external -isystem /opt/homebrew/include -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness -Werror -g -std=gnu++17 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk -fPIE -fcolor-diagnostics -MD -MT velox/common/base/tests/CMakeFiles/velox_memcpy_meter.dir/Memcpy.cpp.o -MF velox/common/base/tests/CMakeFiles/velox_memcpy_meter.dir/Memcpy.cpp.o.d -o velox/common/base/tests/CMakeFiles/velox_memcpy_meter.dir/Memcpy.cpp.o -c /Users/sanskarm/OSS/velox/velox/common/base/tests/Memcpy.cpp
/Users/sanskarm/OSS/velox/velox/common/base/tests/Memcpy.cpp:61:10: error: 'init' is deprecated: Use the RAII version Init [-Werror,-Wdeprecated-declarations]
  folly::init(&argc, &argv);
         ^
/opt/homebrew/include/folly/init/Init.h:99:3: note: 'init' has been explicitly marked deprecated here
[[deprecated("Use the RAII version Init")]] void init(
  ^
1 error generated.
s0nskar commented 10 months ago

Also, it does looks like that setup scripts are installing 8.0.1 fmt, is there any way to override this version?

+ run_and_time install_fmt
+ install_fmt
+ github_checkout fmtlib/fmt 8.0.1
+ local REPO=fmtlib/fmt
+ shift
+ local VERSION=8.0.1
+ shift
+ local GIT_CLONE_PARAMS=
++ basename fmtlib/fmt
+ local DIRNAME=fmt
+ cd /Users/sanskarm/OSS/velox
+ '[' -z fmt ']'
+ '[' -d fmt ']'
+ '[' '!' -d fmt ']'
+ git clone -q -b 8.0.1 https://github.com/fmtlib/fmt.git
Note: switching to 'd141cdbeb0fb422a3fb7173b285fd38e0d1772dc'.
kgpai commented 10 months ago

So from your cmake logs you are using the system fmt , which is not 8.0.1 (unless the PR linked above lands you still need 8.0.1) . So the right thing to do on your end is to build with fmt_SOURCE=BUNDLED make debug. Now coming to the error you are seeing there, it looks like again cmake is picking up some folly version installed on your system that doesnt work with velox, so can you try fmt_SOURCE=BUNDLED folly_SOURCE=BUNDLED make debug ?

s0nskar commented 10 months ago

So tried two things, both failed with different errors, attaching logs

  1. fmt_SOURCE=BUNDLED folly_SOURCE=BUNDLED make debug 

    [621/2006] Linking CXX executable _deps/folly-build/folly/logging/example/logging_example
FAILED: _deps/folly-build/folly/logging/example/logging_example 
: && /Library/Developer/CommandLineTools/usr/bin/c++ -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -fvisibility=hidden -fvisibility-inlines-hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness -g -g -Wall -Wextra -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-flat_namespace _deps/folly-build/folly/logging/example/CMakeFiles/logging_example.dir/main.cpp.o -o _deps/folly-build/folly/logging/example/logging_example  -Wl,-rpath,/opt/homebrew/lib  _deps/folly-build/folly/logging/example/liblogging_example_lib.a  _deps/folly-build/libfolly.a  _deps/fmt-build/libfmtd.a  /opt/homebrew/lib/libboost_context-mt.dylib  /opt/homebrew/lib/libboost_filesystem-mt.dylib  /opt/homebrew/lib/libboost_atomic-mt.dylib  /opt/homebrew/lib/libboost_program_options-mt.dylib  /opt/homebrew/lib/libboost_regex-mt.dylib  /opt/homebrew/lib/libboost_system-mt.dylib  /opt/homebrew/lib/libboost_thread-mt.dylib  /opt/homebrew/lib/libdouble-conversion.dylib  /opt/homebrew/lib/libgflags.2.2.2.dylib  /opt/homebrew/lib/libglog.dylib  /opt/homebrew/lib/libevent.dylib  /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/usr/lib/libz.tbd  /opt/homebrew/lib/libssl.dylib  /opt/homebrew/lib/libcrypto.dylib  /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/usr/lib/libbz2.tbd  /opt/homebrew/lib/liblz4.dylib  /opt/homebrew/lib/libzstd.dylib  /opt/homebrew/lib/libsnappy.dylib  /opt/homebrew/lib/libsodium.dylib  -lc++abi && :
ld: Undefined symbols:
fmt::v10::detail::assert_fail(char const*, int, char const*), referenced from:
  std::__1::make_unsigned<long>::type fmt::v10::detail::to_unsigned<long>(long) in libfolly.a[9](File.cpp.o)
  std::__1::make_unsigned<int>::type fmt::v10::detail::to_unsigned<int>(int) in libfolly.a[9](File.cpp.o)
  fmt::v10::detail::format_decimal_result<char*> fmt::v10::detail::format_decimal<char, unsigned int>(char*, unsigned int, int) in libfolly.a[9](File.cpp.o)
  char const* fmt::v10::detail::parse_dynamic_spec<char>(char const*, char const*, int&, fmt::v10::detail::arg_ref<char>&, fmt::v10::basic_format_parse_context<char>&) in libfolly.a[24](Singleton.cpp.o)
  char const* fmt::v10::detail::parse_dynamic_spec<char>(char const*, char const*, int&, fmt::v10::detail::arg_ref<char>&, fmt::v10::basic_format_parse_context<char>&) in libfolly.a[24](Singleton.cpp.o)
  fmt::v10::detail::fill_t<char>::operator=(fmt::v10::basic_string_view<char>) in libfolly.a[24](Singleton.cpp.o)
  int fmt::v10::detail::parse_nonnegative_int<char>(char const*&, char const*, int) in libfolly.a[24](Singleton.cpp.o)
  ...
fmt::v10::detail::is_printable(unsigned int), referenced from:
  fmt::v10::detail::needs_escape(unsigned int) in libfolly.a[9](File.cpp.o)
fmt::v10::detail::throw_format_error(char const*), referenced from:
  std::__1::back_insert_iterator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>> fmt::v10::detail::concat<fmt::v10::detail::spec_field<char, int, 1>, fmt::v10::detail::concat<fmt::v10::detail::text<char>, fmt::v10::detail::concat<fmt::v10::detail::spec_field<char, unsigned short, 2>, fmt::v10::detail::text<char>>>>::format<std::__1::back_insert_iterator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, char const*, int, unsigned short>(std::__1::back_insert_iterator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, char const* const&, int const&, unsigned short const&) const in libfolly.a[9](File.cpp.o)
  std::__1::back_insert_iterator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>> fmt::v10::detail::concat<fmt::v10::detail::spec_field<char, int, 1>, fmt::v10::detail::concat<fmt::v10::detail::text<char>, fmt::v10::detail::concat<fmt::v10::detail::spec_field<char, unsigned short, 2>, fmt::v10::detail::text<char>>>>::format<std::__1::back_insert_iterator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, char const*, int, unsigned short>(std::__1::back_insert_iterator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, char const* const&, int const&, unsigned short const&) const in libfolly.a[9](File.cpp.o)
  int fmt::v10::detail::get_dynamic_spec<fmt::v10::detail::width_checker, fmt::v10::basic_format_arg<fmt::v10::basic_format_context<std::__1::back_insert_iterator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, char>>>(fmt::v10::basic_format_arg<fmt::v10::basic_format_context<std::__1::back_insert_iterator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, char>>) in libfolly.a[9](File.cpp.o)
  unsigned long long fmt::v10::detail::width_checker::operator()<int, 0>(int) in libfolly.a[9](File.cpp.o)
  unsigned long long fmt::v10::detail::width_checker::operator()<unsigned int, 0>(unsigned int) in libfolly.a[9](File.cpp.o)
  unsigned long long fmt::v10::detail::width_checker::operator()<long long, 0>(long long) in libfolly.a[9](File.cpp.o)
  unsigned long long fmt::v10::detail::width_checker::operator()<unsigned long long, 0>(unsigned long long) in libfolly.a[9](File.cpp.o)
  ...
fmt::v10::vformat(fmt::v10::basic_string_view<char>, fmt::v10::basic_format_args<fmt::v10::basic_format_context<fmt::v10::appender, char>>), referenced from:
  folly::SingletonVault::addToShutdownLog(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>) in libfolly.a[24](Singleton.cpp.o)
  folly::CacheLocality::readFromSysfsTree(std::__1::function<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>)> const&) in libfolly.a[40](CacheLocality.cpp.o)
std::__1::locale fmt::v10::detail::locale_ref::get<std::__1::locale>() const, referenced from:
  fmt::v10::detail::get_locale::get_locale(bool, fmt::v10::detail::locale_ref) in libfolly.a[24](Singleton.cpp.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[632/2006] Building CXX object _deps/duckdb-build/src/common/CMakeFiles/duckdb_common.dir/ub_duckdb_common.cpp.o
ninja: build stopped: subcommand failed.

  2. CPU_TARGET="arm64" fmt_SOURCE=BUNDLED folly_SOURCE=BUNDLED make debug

    FAILED: velox/buffer/tests/CMakeFiles/velox_buffer_test.dir/BufferTest.cpp.o 
/opt/homebrew/bin/ccache /Library/Developer/CommandLineTools/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFMT_LOCALE -DFOLLY_HAVE_INT128_T=1 -DGFLAGS_IS_A_DLL=0 -I/Users/sanskarm/OSS/velox/_build/debug/_deps/protobuf-src/src -I/Users/sanskarm/OSS/velox/. -I/Users/sanskarm/OSS/velox/velox/external/xxhash -I/Users/sanskarm/OSS/velox/_build/debug/_deps/folly-src -I/Users/sanskarm/OSS/velox/_build/debug/_deps/folly-build -I/Users/sanskarm/OSS/velox/_build/debug/_deps/fmt-src/include -I/Users/sanskarm/OSS/velox/_build/debug/_deps/xsimd-src/include -isystem /Users/sanskarm/OSS/velox/velox -isystem /Users/sanskarm/OSS/velox/velox/external -isystem /opt/homebrew/include -isystem /usr/local/include -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/googletest/include -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/googletest -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/googlemock/include -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/googlemock -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src/include -isystem /Users/sanskarm/OSS/velox/_build/debug/_deps/gtest-src -mcpu=apple-m1+crc -std=c++17 -fvisibility=hidden -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis          -Wno-mismatched-tags          -Wno-nullability-completeness -Werror -g -std=gnu++17 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk -fPIE -fcolor-diagnostics -MD -MT velox/buffer/tests/CMakeFiles/velox_buffer_test.dir/BufferTest.cpp.o -MF velox/buffer/tests/CMakeFiles/velox_buffer_test.dir/BufferTest.cpp.o.d -o velox/buffer/tests/CMakeFiles/velox_buffer_test.dir/BufferTest.cpp.o -c /Users/sanskarm/OSS/velox/velox/buffer/tests/BufferTest.cpp
In file included from /Users/sanskarm/OSS/velox/velox/buffer/tests/BufferTest.cpp:17:
In file included from /Users/sanskarm/OSS/velox/./velox/buffer/Buffer.h:23:
In file included from /Users/sanskarm/OSS/velox/./velox/common/base/CheckedArithmetic.h:22:
In file included from /Users/sanskarm/OSS/velox/./velox/common/base/Exceptions.h:26:
In file included from /Users/sanskarm/OSS/velox/_build/debug/_deps/folly-src/folly/Conv.h:128:
/Users/sanskarm/OSS/velox/_build/debug/_deps/folly-src/folly/Range.h:197:12: error: 'char_traits<unsigned char>' is deprecated: char_traits<T> for T not equal to char, wchar_t, char8_t, char16_t or char32_t is non-standard and is provided for a temporary period. It will be removed in LLVM 18, so please migrate off of it. [-Werror,-Wdeprecated-declarations]
  std::char_traits<typename std::remove_const<value_type>::type>;
       ^
/Users/sanskarm/OSS/velox/_build/debug/_deps/folly-src/folly/Range.h:110:20: note: in instantiation of template class 'folly::Range<const unsigned char *>' requested here
const typename Range<Iter>::value_type& needle);
               ^
/Users/sanskarm/OSS/velox/_build/debug/_deps/folly-src/folly/Range.h:1495:15: note: while substituting deduced template arguments into function template 'qfind' [with Iter = const unsigned char *]
inline size_t qfind(
          ^
/Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/__string/char_traits.h:79:8: note: 'char_traits<unsigned char>' has been explicitly marked deprecated here
struct _LIBCPP_DEPRECATED_("char_traits<T> for T not equal to char, wchar_t, char8_t, char16_t or char32_t is non-standard and is provided for a temporary period. It will be removed in LLVM 18, so please migrate off of it.")
   ^
/Library/Developer/CommandLineTools/SDKs/MacOSX14.2.sdk/usr/include/c++/v1/__config:836:53: note: expanded from macro '_LIBCPP_DEPRECATED_'
#      define _LIBCPP_DEPRECATED_(m) __attribute__((deprecated(m)))
                                                ^
1 error generated.
majetideepak commented 10 months ago

If you have another fmt version in your build path it could still conflict with folly even if you bundle fmt. Looking at the error below, this does seem to be the case. Can you delete your system dependencies? fmt::v10::detail::assert_fail(char const*, int, char const*), referenced from: std::__1::make_unsigned<long>::type fmt::v10::detail::to_unsigned<long>(long) in libfolly.a[9]