search

facebookincubator / velox

A composable and fully extensible C++ execution engine library for data management systems.
https://velox-lib.io/
Apache License 2.0
3.49k stars 1.15k forks source link

Folly json headers not found in linux and macos CI jobs #9194

Closed karteekmurthys closed 7 months ago

karteekmurthys commented 7 months ago

Bug description

Here are the list of jobs failing with the same error: linux CI job macos-14 macos-13

I am able to build on local macbook (14.3.1 (23D60)).

System information

Linux and MacOS 14 and 13.

Relevant logs

[1199/2476] Building CXX object velox/common/base/tests/CMakeFiles/velox_base_test.dir/SimdUtilTest.cpp.o
[1200/2476] Building CXX object velox/common/base/tests/CMakeFiles/velox_base_test.dir/BitUtilTest.cpp.o
FAILED: velox/common/base/tests/CMakeFiles/velox_base_test.dir/BitUtilTest.cpp.o 
/usr/bin/ccache /usr/bin/c++ -DBOOST_ATOMIC_NO_LIB -DBOOST_ATOMIC_STATIC_LINK -DBOOST_CHRONO_NO_LIB -DBOOST_CHRONO_STATIC_LINK -DBOOST_CONTAINER_NO_LIB -DBOOST_CONTAINER_STATIC_LINK -DBOOST_CONTEXT_EXPORT="" -DBOOST_CONTEXT_NO_LIB="" -DBOOST_CONTEXT_STATIC_LINK="" -DBOOST_DATE_TIME_NO_LIB -DBOOST_DATE_TIME_STATIC_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_FILESYSTEM_STATIC_LINK=1 -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_PROGRAM_OPTIONS_STATIC_LINK -DBOOST_RANDOM_NO_LIB -DBOOST_RANDOM_STATIC_LINK -DBOOST_THREAD_NO_LIB -DBOOST_THREAD_STATIC_LINK -DBOOST_THREAD_USE_LIB -DFOLLY_HAVE_INT128_T=1 -DGFLAGS_IS_A_DLL=0 -DGLOG_CUSTOM_PREFIX_SUPPORT -I/home/runner/work/velox/velox/_build/debug/_deps/protobuf-src/src -I/home/runner/work/velox/velox/. -I/home/runner/work/velox/velox/velox/external/xxhash -I/home/runner/work/velox/velox/_build/debug/_deps/glog-src/src -I/home/runner/work/velox/velox/_build/debug/_deps/xsimd-src/include -isystem /home/runner/work/velox/velox/velox -isystem /home/runner/work/velox/velox/velox/external -isystem /home/runner/work/velox/velox/_build/debug/_deps/folly-src -isystem /home/runner/work/velox/velox/_build/debug/_deps/folly-build -isystem /home/runner/work/velox/velox/_build/debug/_deps/gflags-build/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/glog-build -isystem /usr/include/libdwarf -isystem /home/runner/work/velox/velox/_build/debug/_deps/fmt-src/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/atomic/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/assert/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/config/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/type_traits/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/static_assert/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/context/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/core/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/throw_exception/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/mp11/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/pool/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/integer/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/winapi/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/predef/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/smart_ptr/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/move/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/date_time/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/algorithm/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/array/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/bind/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/concept_check/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/preprocessor/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/exception/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/tuple/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/function/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/iterator/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/detail/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/function_types/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/mpl/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/utility/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/io/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/fusion/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/container_hash/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/describe/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/typeof/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/functional/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/optional/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/range/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/conversion/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/regex/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/unordered/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/lexical_cast/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/container/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/intrusive/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/numeric/conversion/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/tokenizer/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/filesystem/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/system/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/variant2/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/program_options/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/any/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/type_index/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/thread/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/chrono/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/ratio/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/headers/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/crc/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/circular_buffer/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/math/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/random/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/dynamic_bitset/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/multi_index/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/multiprecision/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/uuid/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/tti/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/boost-src/libs/variant/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/gtest-src/googletest/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/gtest-src/googletest -isystem /home/runner/work/velox/velox/_build/debug/_deps/gtest-src/googlemock/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/gtest-src/googlemock -isystem /home/runner/work/velox/velox/_build/debug/_deps/gtest-src/include -isystem /home/runner/work/velox/velox/_build/debug/_deps/gtest-src -mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-implicit-fallthrough          -Wno-empty-body          -Wno-class-memaccess          -Wno-comment          -Wno-int-in-bool-context          -Wno-redundant-move          -Wno-array-bounds          -Wno-maybe-uninitialized          -Wno-unused-result          -Wno-format-overflow          -Wno-strict-aliasing          -Wno-type-limits          -Wno-stringop-overflow          -Wno-stringop-overread          -Wno-return-type -Werror -g -std=gnu++17 -fPIE -fdiagnostics-color=always -MD -MT velox/common/base/tests/CMakeFiles/velox_base_test.dir/BitUtilTest.cpp.o -MF velox/common/base/tests/CMakeFiles/velox_base_test.dir/BitUtilTest.cpp.o.d -o velox/common/base/tests/CMakeFiles/velox_base_test.dir/BitUtilTest.cpp.o -c /home/runner/work/velox/velox/velox/common/base/tests/BitUtilTest.cpp
In file included from /home/runner/work/velox/velox/velox/common/base/tests/BitUtilTest.cpp:19:
/home/runner/work/velox/velox/./velox/type/HugeInt.h:16:10: fatal error: folly/json/dynamic.h: No such file or directory
   16 | #include <folly/json/dynamic.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
[1201/2476] Building CXX object velox/buffer/CMakeFiles/velox_buffer.dir/StringViewBufferHolder.cpp.o
      |          ^~~~~~~~~~~

MACOS-14:

-Wno-mismatched-tags          -Wno-nullability-completeness -Werror -g -std=gnu++17 -arch arm64 -isysroot /Applications/Xcode_15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -fPIC -MD -MT velox/buffer/CMakeFiles/velox_buffer.dir/StringViewBufferHolder.cpp.o -MF velox/buffer/CMakeFiles/velox_buffer.dir/StringViewBufferHolder.cpp.o.d -o velox/buffer/CMakeFiles/velox_buffer.dir/StringViewBufferHolder.cpp.o -c /Users/runner/work/velox/velox/velox/buffer/StringViewBufferHolder.cpp
In file included from /Users/runner/work/velox/velox/velox/buffer/StringViewBufferHolder.cpp:17:
In file included from /Users/runner/work/velox/velox/./velox/buffer/StringViewBufferHolder.h:20:
/Users/runner/work/velox/velox/./velox/type/StringView.h:26:10: fatal error: 'folly/json/dynamic.h' file not found
#include <folly/json/dynamic.h>
         ^~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[984/2413] Building CXX object velox/buffer/tests/CMakeFiles/velox_buffer_test.dir/BufferTest.cpp.o
FAILED: velox/buffer/tests/CMakeFiles/velox_buffer_test.dir/BufferTest.cpp.o
karteekmurthys commented 7 months ago

@Gownta I see a recent PR: https://github.com/facebookincubator/velox/pull/8931/files related to folly json was added. Is this causing the issue?

mbasmanova commented 7 months ago

CC: @kevinwilfong @kgpai @assignUser @majetideepak

mbasmanova commented 7 months ago

Reverting to unblock: https://github.com/facebookincubator/velox/pull/9197

assignUser commented 7 months ago

Upgrading folly would make this work, so #8950 would be the best place to reintroduce this change.