search

apache / arrow

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
https://arrow.apache.org/
Apache License 2.0
14.3k stars 3.47k forks source link

[Java] Failed when building JNI libraries (*.so on Linux) for arrow-java-dataset, has anyone ever see the same question? #41191

Closed jaynewchen closed 5 months ago

jaynewchen commented 5 months ago

Describe the usage question you have. Please include as many useful details as possible.

My purpose：To build arrow java dataset module (including the JNI part)

I hava built the JNI C Data Interface library successful in a x86 redhat Linux machine，followed the doc：https://arrow.apache.org/docs/developers/java/building.html#building-java-jni-modules

with these commands:

cd arrow/java
mvn generate-resources -Pgenerate-libs-cdata-all-os -N

But when I run these commands to build all JNI libraries except the JNI C Data Interface library (including dataset module)，I get this output：

mvn generate-resources -Pgenerate-libs-jni-macos-linux -N
INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: linux
[INFO] os.detected.arch: x86_64
[INFO] os.detected.bitness: 64
[INFO] os.detected.version: 3.10
[INFO] os.detected.version.major: 3
[INFO] os.detected.version.minor: 10
[INFO] os.detected.release: euleros
[INFO] os.detected.release.version: 2.0
[INFO] os.detected.release.like.euleros: true
[INFO] os.detected.release.like.rhel: true
[INFO] os.detected.release.like.fedora: true
[INFO] os.detected.release.like.centos: true
[INFO] os.detected.classifier: linux-x86_64
[INFO]
[INFO] ------------------< org.apache.arrow:arrow-java-root >------------------
[INFO] Building Apache Arrow Java Root POM 14.0.1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-checks) @ arrow-java-root ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 62 implicit excludes (use -debug for more details).
[INFO] 39 explicit excludes (use -debug for more details).
[INFO] 1196 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 1193 licenses.
[INFO]
[INFO] --- maven-checkstyle-plugin:3.1.0:check (validate) @ arrow-java-root ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- git-commit-id-plugin:4.0.5:revision (for-jars) @ arrow-java-root ---
[INFO]
[INFO] --- git-commit-id-plugin:4.0.5:revision (for-source-tarball) @ arrow-java-root ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ arrow-java-root ---
[INFO]
[INFO] --- exec-maven-plugin:3.1.0:exec (jni-cpp-cmake) @ arrow-java-root ---
-- Building using CMake version: 3.21.0
-- Arrow version: 14.0.1 (full: '14.0.1')
-- Arrow SO version: 1400 (full: 1400.1.0)
-- clang-tidy 14 not found
-- clang-format 14 not found
-- Could NOT find ClangTools (missing: CLANG_FORMAT_BIN CLANG_TIDY_BIN)
-- infer not found
-- Could NOT find Python3 (missing: Python3_EXECUTABLE Interpreter)
    Reason given by package:
        Interpreter: Cannot use the interpreter "/opt/shangh/ck_tools/python-3.9.11/bin/python3.9"

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
-- Found cpplint executable at /srv/cjx/project/arrow-release-14.0.1-rc0/cpp/build-support/cpplint.py
-- System processor: x86_64
-- Arrow build warning level: PRODUCTION
-- Using ld linker
-- Build Type: RELEASE
-- Using BUNDLED approach to find dependencies
-- ARROW_ABSL_BUILD_VERSION: 20211102.0
-- ARROW_ABSL_BUILD_SHA256_CHECKSUM: dcf71b9cba8dc0ca9940c4b316a0c796be8fab42b070bb6b7cab62b48f0e66c4
-- ARROW_AWS_C_AUTH_BUILD_VERSION: v0.6.22
-- ARROW_AWS_C_AUTH_BUILD_SHA256_CHECKSUM: 691a6b4418afcd3dc141351b6ad33fccd8e3ff84df0e9e045b42295d284ee14c
-- ARROW_AWS_C_CAL_BUILD_VERSION: v0.5.20
-- ARROW_AWS_C_CAL_BUILD_SHA256_CHECKSUM: acc352359bd06f8597415c366cf4ec4f00d0b0da92d637039a73323dd55b6cd0
-- ARROW_AWS_C_COMMON_BUILD_VERSION: v0.8.9
-- ARROW_AWS_C_COMMON_BUILD_SHA256_CHECKSUM: 2f3fbaf7c38eae5a00e2a816d09b81177f93529ae8ba1b82dc8f31407565327a
-- ARROW_AWS_C_COMPRESSION_BUILD_VERSION: v0.2.16
-- ARROW_AWS_C_COMPRESSION_BUILD_SHA256_CHECKSUM: 044b1dbbca431a07bde8255ef9ec443c300fc60d4c9408d4b862f65e496687f4
-- ARROW_AWS_C_EVENT_STREAM_BUILD_VERSION: v0.2.18
-- ARROW_AWS_C_EVENT_STREAM_BUILD_SHA256_CHECKSUM: 310ca617f713bf664e4c7485a3d42c1fb57813abd0107e49790d107def7cde4f
-- ARROW_AWS_C_HTTP_BUILD_VERSION: v0.7.3
-- ARROW_AWS_C_HTTP_BUILD_SHA256_CHECKSUM: 07e16c6bf5eba6f0dea96b6f55eae312a7c95b736f4d2e4a210000f45d8265ae
-- ARROW_AWS_C_IO_BUILD_VERSION: v0.13.14
-- ARROW_AWS_C_IO_BUILD_SHA256_CHECKSUM: 12b66510c3d9a4f7e9b714e9cfab2a5bf835f8b9ce2f909d20ae2a2128608c71
-- ARROW_AWS_C_MQTT_BUILD_VERSION: v0.8.4
-- ARROW_AWS_C_MQTT_BUILD_SHA256_CHECKSUM: 232eeac63e72883d460c686a09b98cdd811d24579affac47c5c3f696f956773f
-- ARROW_AWS_C_S3_BUILD_VERSION: v0.2.3
-- ARROW_AWS_C_S3_BUILD_SHA256_CHECKSUM: a00b3c9f319cd1c9aa2c3fa15098864df94b066dcba0deaccbb3caa952d902fe
-- ARROW_AWS_C_SDKUTILS_BUILD_VERSION: v0.1.6
-- ARROW_AWS_C_SDKUTILS_BUILD_SHA256_CHECKSUM: 8a2951344b2fb541eab1e9ca17c18a7fcbfd2aaff4cdd31d362d1fad96111b91
-- ARROW_AWS_CHECKSUMS_BUILD_VERSION: v0.1.13
-- ARROW_AWS_CHECKSUMS_BUILD_SHA256_CHECKSUM: 0f897686f1963253c5069a0e495b85c31635ba146cd3ac38cc2ea31eaf54694d
-- ARROW_AWS_CRT_CPP_BUILD_VERSION: v0.18.16
-- ARROW_AWS_CRT_CPP_BUILD_SHA256_CHECKSUM: 9e69bc1dc4b50871d1038aa9ff6ddeb4c9b28f7d6b5e5b1b69041ccf50a13483
-- ARROW_AWS_LC_BUILD_VERSION: v1.3.0
-- ARROW_AWS_LC_BUILD_SHA256_CHECKSUM: ae96a3567161552744fc0cae8b4d68ed88b1ec0f3d3c98700070115356da5a37
-- ARROW_AWSSDK_BUILD_VERSION: 1.10.55
-- ARROW_AWSSDK_BUILD_SHA256_CHECKSUM: 2d552fb1a84bef4a9b65e34aa7031851ed2aef5319e02cc6e4cb735c48aa30de
-- ARROW_AZURE_SDK_BUILD_VERSION: azure-core_1.10.3
-- ARROW_AZURE_SDK_BUILD_SHA256_CHECKSUM: dd624c2f86adf474d2d0a23066be6e27af9cbd7e3f8d9d8fd7bf981e884b7b48
-- ARROW_BOOST_BUILD_VERSION: 1.81.0
-- ARROW_BOOST_BUILD_SHA256_CHECKSUM: 9e0ffae35528c35f90468997bc8d99500bf179cbae355415a89a600c38e13574
-- ARROW_BROTLI_BUILD_VERSION: v1.0.9
-- ARROW_BROTLI_BUILD_SHA256_CHECKSUM: f9e8d81d0405ba66d181529af42a3354f838c939095ff99930da6aa9cdf6fe46
-- ARROW_BZIP2_BUILD_VERSION: 1.0.8
-- ARROW_BZIP2_BUILD_SHA256_CHECKSUM: ab5a03176ee106d3f0fa90e381da478ddae405918153cca248e682cd0c4a2269
-- ARROW_CARES_BUILD_VERSION: 1.17.2
-- ARROW_CARES_BUILD_SHA256_CHECKSUM: 4803c844ce20ce510ef0eb83f8ea41fa24ecaae9d280c468c582d2bb25b3913d
-- ARROW_CRC32C_BUILD_VERSION: 1.1.2
-- ARROW_CRC32C_BUILD_SHA256_CHECKSUM: ac07840513072b7fcebda6e821068aa04889018f24e10e46181068fb214d7e56
-- ARROW_GBENCHMARK_BUILD_VERSION: v1.7.1
-- ARROW_GBENCHMARK_BUILD_SHA256_CHECKSUM: 6430e4092653380d9dc4ccb45a1e2dc9259d581f4866dc0759713126056bc1d7
-- ARROW_GFLAGS_BUILD_VERSION: v2.2.2
-- ARROW_GFLAGS_BUILD_SHA256_CHECKSUM: 34af2f15cf7367513b352bdcd2493ab14ce43692d2dcd9dfc499492966c64dcf
-- ARROW_GLOG_BUILD_VERSION: v0.5.0
-- ARROW_GLOG_BUILD_SHA256_CHECKSUM: eede71f28371bf39aa69b45de23b329d37214016e2055269b3b5e7cfd40b59f5
-- ARROW_GOOGLE_CLOUD_CPP_BUILD_VERSION: v2.12.0
-- ARROW_GOOGLE_CLOUD_CPP_BUILD_SHA256_CHECKSUM: 8cda870803925c62de8716a765e03eb9d34249977e5cdb7d0d20367e997a55e2
-- ARROW_GRPC_BUILD_VERSION: v1.46.3
-- ARROW_GRPC_BUILD_SHA256_CHECKSUM: d6cbf22cb5007af71b61c6be316a79397469c58c82a942552a62e708bce60964
-- ARROW_GTEST_BUILD_VERSION: 1.11.0
-- ARROW_GTEST_BUILD_SHA256_CHECKSUM: b4870bf121ff7795ba20d20bcdd8627b8e088f2d1dab299a031c1034eddc93d5
-- ARROW_JEMALLOC_BUILD_VERSION: 5.3.0
-- ARROW_JEMALLOC_BUILD_SHA256_CHECKSUM: 2db82d1e7119df3e71b7640219b6dfe84789bc0537983c3b7ac4f7189aecfeaa
-- ARROW_LZ4_BUILD_VERSION: v1.9.4
-- ARROW_LZ4_BUILD_SHA256_CHECKSUM: 0b0e3aa07c8c063ddf40b082bdf7e37a1562bda40a0ff5272957f3e987e0e54b
-- ARROW_MIMALLOC_BUILD_VERSION: v2.0.6
-- ARROW_MIMALLOC_BUILD_SHA256_CHECKSUM: 9f05c94cc2b017ed13698834ac2a3567b6339a8bde27640df5a1581d49d05ce5
-- ARROW_NLOHMANN_JSON_BUILD_VERSION: v3.10.5
-- ARROW_NLOHMANN_JSON_BUILD_SHA256_CHECKSUM: 5daca6ca216495edf89d167f808d1d03c4a4d929cef7da5e10f135ae1540c7e4
-- ARROW_OPENTELEMETRY_BUILD_VERSION: v1.8.1
-- ARROW_OPENTELEMETRY_BUILD_SHA256_CHECKSUM: 3d640201594b07f08dade9cd1017bd0b59674daca26223b560b9bb6bf56264c2
-- ARROW_OPENTELEMETRY_PROTO_BUILD_VERSION: v0.17.0
-- ARROW_OPENTELEMETRY_PROTO_BUILD_SHA256_CHECKSUM: f269fbcb30e17b03caa1decd231ce826e59d7651c0f71c3b28eb5140b4bb5412
-- ARROW_ORC_BUILD_VERSION: 1.9.0
-- ARROW_ORC_BUILD_SHA256_CHECKSUM: 0dca8bbccdb2ee87e59ba964933436beebd02ea78c4134424828a8127fbc4faa
-- ARROW_PROTOBUF_BUILD_VERSION: v21.3
-- ARROW_PROTOBUF_BUILD_SHA256_CHECKSUM: 2f723218f6cb709ae4cdc4fb5ed56a5951fc5d466f0128ce4c946b8c78c8c49f
-- ARROW_RAPIDJSON_BUILD_VERSION: 232389d4f1012dddec4ef84861face2d2ba85709
-- ARROW_RAPIDJSON_BUILD_SHA256_CHECKSUM: b9290a9a6d444c8e049bd589ab804e0ccf2b05dc5984a19ed5ae75d090064806
-- ARROW_RE2_BUILD_VERSION: 2022-06-01
-- ARROW_RE2_BUILD_SHA256_CHECKSUM: f89c61410a072e5cbcf8c27e3a778da7d6fd2f2b5b1445cd4f4508bee946ab0f
-- ARROW_SNAPPY_BUILD_VERSION: 1.1.10
-- ARROW_SNAPPY_BUILD_SHA256_CHECKSUM: 49d831bffcc5f3d01482340fe5af59852ca2fe76c3e05df0e67203ebbe0f1d90
-- ARROW_SUBSTRAIT_BUILD_VERSION: v0.27.0
-- ARROW_SUBSTRAIT_BUILD_SHA256_CHECKSUM: 4ed375f69d972a57fdc5ec406c17003a111831d8640d3f1733eccd4b3ff45628
-- ARROW_S2N_TLS_BUILD_VERSION: v1.3.35
-- ARROW_S2N_TLS_BUILD_SHA256_CHECKSUM: 9d32b26e6bfcc058d98248bf8fc231537e347395dd89cf62bb432b55c5da990d
-- ARROW_THRIFT_BUILD_VERSION: 0.16.0
-- ARROW_THRIFT_BUILD_SHA256_CHECKSUM: f460b5c1ca30d8918ff95ea3eb6291b3951cf518553566088f3f2be8981f6209
-- ARROW_UCX_BUILD_VERSION: 1.12.1
-- ARROW_UCX_BUILD_SHA256_CHECKSUM: 9bef31aed0e28bf1973d28d74d9ac4f8926c43ca3b7010bd22a084e164e31b71
-- ARROW_UTF8PROC_BUILD_VERSION: v2.7.0
-- ARROW_UTF8PROC_BUILD_SHA256_CHECKSUM: 4bb121e297293c0fd55f08f83afab6d35d48f0af4ecc07523ad8ec99aa2b12a1
-- ARROW_XSIMD_BUILD_VERSION: 9.0.1
-- ARROW_XSIMD_BUILD_SHA256_CHECKSUM: b1bb5f92167fd3a4f25749db0be7e61ed37e0a5d943490f3accdcd2cd2918cc0
-- ARROW_ZLIB_BUILD_VERSION: 1.2.13
-- ARROW_ZLIB_BUILD_SHA256_CHECKSUM: b3a24de97a8fdbc835b9833169501030b8977031bcb54b3b3ac13740f846ab30
-- ARROW_ZSTD_BUILD_VERSION: 1.5.5
-- ARROW_ZSTD_BUILD_SHA256_CHECKSUM: 9c4396cc829cfae319a6e2615202e82aad41372073482fce286fac78646d3ee4
-- Boost include dir:
-- Providing CMake module for OpenSSLAlt as part of Arrow CMake package
-- Found OpenSSL Crypto Library: /usr/local/openssl/lib/libcrypto.a
-- Building with OpenSSL (Version: 1.1.1k) support
-- Building Apache Thrift from source
-- Building Protocol Buffers from source
-- Found protoc: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/protobuf_ep-install/bin/protoc
-- Building Substrait from source
-- Building jemalloc from source
-- Building RapidJSON from source
-- Building xsimd from source
-- Building ZLIB from source
-- Building RE2 from source
-- Building utf8proc from source
-- Found hdfs.h at: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp/thirdparty/hadoop/include/hdfs.h
-- Building AWS C++ SDK from source
-- Found AWS SDK headers: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/awssdk_ep-install/include
-- Found AWS SDK libraries: aws-cpp-sdk-identity-management;aws-cpp-sdk-sts;aws-cpp-sdk-cognito-identity;aws-cpp-sdk-s3;aws-cpp-sdk-core;AWS::aws-crt-cpp;AWS::aws-c-s3;AWS::aws-c-auth;AWS:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         :aws-c-mqtt;AWS::aws-c-http;AWS::aws-c-compression;AWS::aws-c-sdkutils;AWS::aws-c-event-stream;AWS::aws-c-io;AWS::aws-c-cal;AWS::aws-checksums;AWS::aws-c-common;AWS::s2n-tls
-- All bundled static libraries: thrift::thrift;arrow::protobuf::libprotobuf;substrait;jemalloc::jemalloc;ZLIB::ZLIB;re2::re2;utf8proc::utf8proc;aws-cpp-sdk-identity-management;aws-cpp-sdk                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         -sts;aws-cpp-sdk-cognito-identity;aws-cpp-sdk-s3;aws-cpp-sdk-core;AWS::aws-crt-cpp;AWS::aws-c-s3;AWS::aws-c-auth;AWS::aws-c-mqtt;AWS::aws-c-http;AWS::aws-c-compression;AWS::aws-c-sdkutils;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         AWS::aws-c-event-stream;AWS::aws-c-io;AWS::aws-c-cal;AWS::aws-checksums;AWS::aws-c-common;AWS::s2n-tls
-- CMAKE_C_FLAGS:   -Wall -fno-semantic-interposition -msse4.2
-- CMAKE_CXX_FLAGS:  -Wno-noexcept-type -Wno-subobject-linkage  -fdiagnostics-color=always  -Wall -fno-semantic-interposition -msse4.2
-- CMAKE_C_FLAGS_RELEASE: -O3 -DNDEBUG -O2 -ftree-vectorize
-- CMAKE_CXX_FLAGS_RELEASE: -O3 -DNDEBUG -O2 -ftree-vectorize
-- Creating bundled static library target arrow_bundled_dependencies at /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/release/libarrow_bundled_dependencies.a
CMake Error at src/arrow/CMakeLists.txt:715 (target_compile_features):
  target_compile_features no known features for CXX compiler

  "GNU"

  version 9.4.0.

CMake Error at src/arrow/CMakeLists.txt:715 (target_compile_features):
  target_compile_features no known features for CXX compiler

  "GNU"

  version 9.4.0.

-- ---------------------------------------------------------------------
-- Arrow version:                                 14.0.1
--
-- Build configuration summary:
--   Generator: Unix Makefiles
--   Build type: RELEASE
--   Source directory: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp
--   Install prefix: /srv/cjx/project/arrow-release-14.0.1-rc0/java-dist
--
-- Compile and link options:
--
--   ARROW_CXXFLAGS="" [default=""]
--       Compiler flags to append when compiling Arrow
--   ARROW_BUILD_STATIC=ON [default=ON]
--       Build static libraries
--   ARROW_BUILD_SHARED=OFF [default=ON]
--       Build shared libraries
--   ARROW_PACKAGE_KIND="" [default=""]
--       Arbitrary string that identifies the kind of package
--       (for informational purposes)
--   ARROW_GIT_ID="" [default=""]
--       The Arrow git commit id (if any)
--   ARROW_GIT_DESCRIPTION="" [default=""]
--       The Arrow git commit description (if any)
--   ARROW_NO_DEPRECATED_API=OFF [default=OFF]
--       Exclude deprecated APIs from build
--   ARROW_POSITION_INDEPENDENT_CODE=ON [default=ON]
--       Whether to create position-independent target
--   ARROW_USE_CCACHE=ON [default=ON]
--       Use ccache when compiling (if available)
--   ARROW_USE_SCCACHE=ON [default=ON]
--       Use sccache when compiling (if available),
--       takes precedence over ccache if a storage backend is configured
--   ARROW_USE_LD_GOLD=OFF [default=OFF]
--       Use ld.gold for linking on Linux (if available)
--   ARROW_USE_PRECOMPILED_HEADERS=OFF [default=OFF]
--       Use precompiled headers when compiling
--   ARROW_SIMD_LEVEL=SSE4_2 [default=DEFAULT|NONE|SSE4_2|AVX2|AVX512|NEON|SVE|SVE128|SVE256|SVE512]
--       Compile-time SIMD optimization level
--   ARROW_RUNTIME_SIMD_LEVEL=MAX [default=MAX|NONE|SSE4_2|AVX2|AVX512]
--       Max runtime SIMD optimization level
--   ARROW_ALTIVEC=ON [default=ON]
--       Build with Altivec if compiler has support
--   ARROW_RPATH_ORIGIN=OFF [default=OFF]
--       Build Arrow libraries with RATH set to $ORIGIN
--   ARROW_INSTALL_NAME_RPATH=ON [default=ON]
--       Build Arrow libraries with install_name set to @rpath
--   ARROW_GGDB_DEBUG=ON [default=ON]
--       Pass -ggdb flag to debug builds
--   ARROW_WITH_MUSL=OFF [default=OFF]
--       Whether the system libc is musl or not
--   ARROW_ENABLE_THREADING=ON [default=ON]
--       Enable threading in Arrow core
--
-- Test and benchmark options:
--
--   ARROW_BUILD_EXAMPLES=OFF [default=OFF]
--       Build the Arrow examples
--   ARROW_BUILD_TESTS=OFF [default=OFF]
--       Build the Arrow googletest unit tests
--   ARROW_ENABLE_TIMING_TESTS=ON [default=ON]
--       Enable timing-sensitive tests
--   ARROW_BUILD_INTEGRATION=OFF [default=OFF]
--       Build the Arrow integration test executables
--   ARROW_BUILD_BENCHMARKS=OFF [default=OFF]
--       Build the Arrow micro benchmarks
--   ARROW_BUILD_BENCHMARKS_REFERENCE=OFF [default=OFF]
--       Build the Arrow micro reference benchmarks
--   ARROW_BUILD_OPENMP_BENCHMARKS=OFF [default=OFF]
--       Build the Arrow benchmarks that rely on OpenMP
--   ARROW_BUILD_DETAILED_BENCHMARKS=OFF [default=OFF]
--       Build benchmarks that do a longer exploration of performance
--   ARROW_TEST_LINKAGE=static [default=static|shared]
--       Linkage of Arrow libraries with unit tests executables.
--   ARROW_FUZZING=OFF [default=OFF]
--       Build Arrow Fuzzing executables
--   ARROW_LARGE_MEMORY_TESTS=OFF [default=OFF]
--       Enable unit tests which use large memory
--
-- Lint options:
--
--   ARROW_ONLY_LINT=OFF [default=OFF]
--       Only define the lint and check-format targets
--   ARROW_VERBOSE_LINT=OFF [default=OFF]
--       If off, 'quiet' flags will be passed to linting tools
--   ARROW_GENERATE_COVERAGE=OFF [default=OFF]
--       Build with C++ code coverage enabled
--
-- Checks options:
--
--   ARROW_TEST_MEMCHECK=OFF [default=OFF]
--       Run the test suite using valgrind --tool=memcheck
--   ARROW_USE_ASAN=OFF [default=OFF]
--       Enable Address Sanitizer checks
--   ARROW_USE_TSAN=OFF [default=OFF]
--       Enable Thread Sanitizer checks
--   ARROW_USE_UBSAN=OFF [default=OFF]
--       Enable Undefined Behavior sanitizer checks
--
-- Project component options:
--
--   ARROW_ACERO=ON [default=OFF]
--       Build the Arrow Acero Engine Module
--   ARROW_AZURE=OFF [default=OFF]
--       Build Arrow with Azure support (requires the Azure SDK for C++)
--   ARROW_BUILD_UTILITIES=OFF [default=OFF]
--       Build Arrow commandline utilities
--   ARROW_COMPUTE=ON [default=OFF]
--       Build all Arrow Compute kernels
--   ARROW_CSV=ON [default=OFF]
--       Build the Arrow CSV Parser Module
--   ARROW_CUDA=OFF [default=OFF]
--       Build the Arrow CUDA extensions (requires CUDA toolkit)
--   ARROW_DATASET=ON [default=OFF]
--       Build the Arrow Dataset Modules
--   ARROW_FILESYSTEM=ON [default=OFF]
--       Build the Arrow Filesystem Layer
--   ARROW_FLIGHT=OFF [default=OFF]
--       Build the Arrow Flight RPC System (requires GRPC, Protocol Buffers)
--   ARROW_FLIGHT_SQL=OFF [default=OFF]
--       Build the Arrow Flight SQL extension
--   ARROW_GANDIVA=OFF [default=OFF]
--       Build the Gandiva libraries
--   ARROW_GCS=OFF [default=OFF]
--       Build Arrow with GCS support (requires the GCloud SDK for C++)
--   ARROW_HDFS=OFF [default=OFF]
--       Build the Arrow HDFS bridge
--   ARROW_IPC=ON [default=ON]
--       Build the Arrow IPC extensions
--   ARROW_JEMALLOC=ON [default=ON]
--       Build the Arrow jemalloc-based allocator
--   ARROW_JSON=ON [default=OFF]
--       Build Arrow with JSON support (requires RapidJSON)
--   ARROW_MIMALLOC=OFF [default=OFF]
--       Build the Arrow mimalloc-based allocator
--   ARROW_PARQUET=ON [default=OFF]
--       Build the Parquet libraries
--   ARROW_ORC=OFF [default=OFF]
--       Build the Arrow ORC adapter
--   ARROW_PYTHON=OFF [default=OFF]
--       Build some components needed by PyArrow.
--       (This is a deprecated option. Use CMake presets instead.)
--   ARROW_S3=ON [default=OFF]
--       Build Arrow with S3 support (requires the AWS SDK for C++)
--   ARROW_SKYHOOK=OFF [default=OFF]
--       Build the Skyhook libraries
--   ARROW_SUBSTRAIT=ON [default=OFF]
--       Build the Arrow Substrait Consumer Module
--   ARROW_TENSORFLOW=OFF [default=OFF]
--       Build Arrow with TensorFlow support enabled
--   ARROW_TESTING=OFF [default=OFF]
--       Build the Arrow testing libraries
--
-- Thirdparty toolchain options:
--
--   ARROW_DEPENDENCY_SOURCE=BUNDLED [default=AUTO|BUNDLED|SYSTEM|CONDA|VCPKG|BREW]
--       Method to use for acquiring arrow's build dependencies
--   ARROW_VERBOSE_THIRDPARTY_BUILD=OFF [default=OFF]
--       Show output from ExternalProjects rather than just logging to files
--   ARROW_DEPENDENCY_USE_SHARED=OFF [default=ON]
--       Link to shared libraries
--   ARROW_BOOST_USE_SHARED=OFF [default=OFF]
--       Rely on Boost shared libraries where relevant
--   ARROW_BROTLI_USE_SHARED=OFF [default=OFF]
--       Rely on Brotli shared libraries where relevant
--   ARROW_BZ2_USE_SHARED=OFF [default=OFF]
--       Rely on Bz2 shared libraries where relevant
--   ARROW_GFLAGS_USE_SHARED=OFF [default=OFF]
--       Rely on GFlags shared libraries where relevant
--   ARROW_GRPC_USE_SHARED=OFF [default=OFF]
--       Rely on gRPC shared libraries where relevant
--   ARROW_JEMALLOC_USE_SHARED=OFF [default=OFF]
--       Rely on jemalloc shared libraries where relevant
--   ARROW_LLVM_USE_SHARED=OFF [default=OFF]
--       Rely on LLVM shared libraries where relevant
--   ARROW_LZ4_USE_SHARED=OFF [default=OFF]
--       Rely on lz4 shared libraries where relevant
--   ARROW_OPENSSL_USE_SHARED=OFF [default=OFF]
--       Rely on OpenSSL shared libraries where relevant
--   ARROW_PROTOBUF_USE_SHARED=OFF [default=OFF]
--       Rely on Protocol Buffers shared libraries where relevant
--   ARROW_SNAPPY_USE_SHARED=OFF [default=OFF]
--       Rely on snappy shared libraries where relevant
--   ARROW_THRIFT_USE_SHARED=OFF [default=OFF]
--       Rely on thrift shared libraries where relevant
--   ARROW_UTF8PROC_USE_SHARED=OFF [default=OFF]
--       Rely on utf8proc shared libraries where relevant
--   ARROW_ZSTD_USE_SHARED=OFF [default=OFF]
--       Rely on zstd shared libraries where relevant
--   ARROW_USE_GLOG=OFF [default=OFF]
--       Build libraries with glog support for pluggable logging
--   ARROW_WITH_BACKTRACE=ON [default=ON]
--       Build with backtrace support
--   ARROW_WITH_OPENTELEMETRY=OFF [default=OFF]
--       Build libraries with OpenTelemetry support for distributed tracing
--   ARROW_WITH_BROTLI=OFF [default=OFF]
--       Build with Brotli compression
--   ARROW_WITH_BZ2=OFF [default=OFF]
--       Build with BZ2 compression
--   ARROW_WITH_LZ4=OFF [default=OFF]
--       Build with lz4 compression
--   ARROW_WITH_SNAPPY=OFF [default=OFF]
--       Build with Snappy compression
--   ARROW_WITH_ZLIB=ON [default=OFF]
--       Build with zlib compression
--   ARROW_WITH_ZSTD=OFF [default=OFF]
--       Build with zstd compression
--   ARROW_WITH_UCX=OFF [default=OFF]
--       Build with UCX transport for Arrow Flight
--       (only used if ARROW_FLIGHT is ON)
--   ARROW_WITH_UTF8PROC=ON [default=ON]
--       Build with support for Unicode properties using the utf8proc library
--       (only used if ARROW_COMPUTE is ON or ARROW_GANDIVA is ON)
--   ARROW_WITH_RE2=ON [default=ON]
--       Build with support for regular expressions using the re2 library
--       (only used if ARROW_COMPUTE or ARROW_GANDIVA is ON)
--
-- Parquet options:
--
--   PARQUET_MINIMAL_DEPENDENCY=OFF [default=OFF]
--       Depend only on Thirdparty headers to build libparquet.
--       Always OFF if building binaries
--   PARQUET_BUILD_EXECUTABLES=OFF [default=OFF]
--       Build the Parquet executable CLI tools. Requires static libraries to be built.
--   PARQUET_BUILD_EXAMPLES=OFF [default=OFF]
--       Build the Parquet examples. Requires static libraries to be built.
--   PARQUET_REQUIRE_ENCRYPTION=OFF [default=OFF]
--       Build support for encryption. Fail if OpenSSL is not found
--
-- Gandiva options:
--
--   ARROW_GANDIVA_STATIC_LIBSTDCPP=ON [default=OFF]
--       Include -static-libstdc++ -static-libgcc when linking with
--       Gandiva static libraries
--   ARROW_GANDIVA_PC_CXX_FLAGS="" [default=""]
--       Compiler flags to append when pre-compiling Gandiva operations
--
-- Advanced developer options:
--
--   ARROW_EXTRA_ERROR_CONTEXT=OFF [default=OFF]
--       Compile with extra error context (line numbers, code)
--   ARROW_OPTIONAL_INSTALL=OFF [default=OFF]
--       If enabled install ONLY targets that have already been built. Please be
--       advised that if this is enabled 'install' will fail silently on components
--       that have not been built
--   ARROW_GDB_INSTALL_DIR="" [default=""]
--       Use a custom install directory for GDB plugin.
--       In general, you don't need to specify this because the default
--       (CMAKE_INSTALL_FULL_BINDIR on Windows, CMAKE_INSTALL_FULL_LIBDIR otherwise)
--       is reasonable.
--   Outputting build configuration summary to /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/cmake_summary.json
-- Configuring incomplete, errors occurred!
See also "/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeOutput.log".
See also "/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeError.log".
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
    at org.apache.commons.exec.DefaultExecutor.executeInternal (DefaultExecutor.java:404)
    at org.apache.commons.exec.DefaultExecutor.execute (DefaultExecutor.java:166)
    at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:1000)
    at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:947)
    at org.codehaus.mojo.exec.ExecMojo.execute (ExecMojo.java:471)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.045 s
[INFO] Finished at: 2024-04-14T15:09:14+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.1.0:exec (jni-cpp-cmake) on project arrow-java-root: Command execution failed.: Process exited with an error: 1 (Exit v                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         alue: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Firstly, I suspect that it's a pthread problem or openssl problem or missing LLVM dependency，but after trying a while, it is found that the failure is not caused by them.

Here's the output of CMakeError.log：

Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/local/bin/make -f Makefile cmTC_daf7e/fast && /usr/local/bin/make  -f CMakeFiles/cmTC_daf7e.dir/build.make CMakeFiles/cmTC_daf7e.dir/build
make[1]: Entering directory `/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_daf7e.dir/src.c.o
/opt/gcc-9.4.0/bin/gcc-9 -DCMAKE_HAVE_LIBC_PTHREAD  -fPIE -std=gnu11 -o CMakeFiles/cmTC_daf7e.dir/src.c.o -c /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_daf7e
/opt/soft/cmake-3.21.0-linux-x86_64/bin/cmake -E cmake_link_script CMakeFiles/cmTC_daf7e.dir/link.txt --verbose=1
/opt/gcc-9.4.0/bin/gcc-9 CMakeFiles/cmTC_daf7e.dir/src.c.o -o cmTC_daf7e
CMakeFiles/cmTC_daf7e.dir/src.c.o: In function `main':
src.c:(.text+0x2f): undefined reference to `pthread_create'
src.c:(.text+0x3b): undefined reference to `pthread_detach'
src.c:(.text+0x47): undefined reference to `pthread_cancel'
src.c:(.text+0x58): undefined reference to `pthread_join'
src.c:(.text+0x6c): undefined reference to `pthread_atfork'
collect2: error: ld returned 1 exit status
make[1]: *** [cmTC_daf7e] Error 1
make[1]: Leaving directory `/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp'
make: *** [cmTC_daf7e/fast] Error 2

Source file was:
#include <pthread.h>

static void* test_func(void* data)
{
  return data;
}

int main(void)
{
  pthread_t thread;
  pthread_create(&thread, NULL, test_func, NULL);
  pthread_detach(thread);
  pthread_cancel(thread);
  pthread_join(thread, NULL);
  pthread_atfork(NULL, NULL, NULL);
  pthread_exit(NULL);

  return 0;
}

Determining if the _M_ARM64 exist failed with the following output:
Change Dir: /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/local/bin/make -f Makefile cmTC_faa90/fast && /usr/local/bin/make  -f CMakeFiles/cmTC_faa90.dir/build.make CMakeFiles/cmTC_faa90.dir/build
make[1]: Entering directory `/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_faa90.dir/CheckSymbolExists.cxx.o
/opt/gcc-9.4.0/bin/g++-9   -fdiagnostics-color=always  -fPIE -std=c++17 -o CMakeFiles/cmTC_faa90.dir/CheckSymbolExists.cxx.o -c /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx
^[[01m^[[K/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:^[[m^[[K In function ‘^[[01m^[[Kint main(int, char**)^[[m^[[K’:
^[[01m^[[K/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:7:19:^[[m^[[K ^[[01;31m^[[Kerror: ^[[m^[[K‘^[[01m^[[K_M_ARM64^[[m^[[K’ was not declared in this scope
    7 |   return ((int*)(&^[[01;31m^[[K_M_ARM64^[[m^[[K))[argc];
      |                   ^[[01;31m^[[K^~~~~~~~^[[m^[[K
make[1]: *** [CMakeFiles/cmTC_faa90.dir/CheckSymbolExists.cxx.o] Error 1
make[1]: Leaving directory `/srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp'
make: *** [cmTC_faa90/fast] Error 2

File /srv/cjx/project/arrow-release-14.0.1-rc0/cpp-jni/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:
/* */

int main(int argc, char** argv)
{
  (void)argv;
#ifndef _M_ARM64
  return ((int*)(&_M_ARM64))[argc];
#else
  (void)argc;
  return 0;
#endif
}

The arrow version I'm using is 14.0.1.rc0, JDK version is 1.8, and compiler is gcc-9.4.0 and g++-9

Can someone help me? thank you very much

Component(s)

C++, Java

kou commented 5 months ago

Could you try 15.0.2 instead of 14.0.0.1-rc0?

What is your RHEL version?

FYI: Here are Dockerfiles to build our Apache Arrow C++ packages for RHEL8/9:

jaynewchen commented 5 months ago

Could you try 15.0.2 instead of 14.0.0.1-rc0?

What is your RHEL version?

FYI: Here are Dockerfiles to build our Apache Arrow C++ packages for RHEL8/9:

My RHEL version is：

# cat /etc/redhat-release
EulerOS release 2.0 (SP5)

# cat /etc/os-release
NAME="EulerOS"
VERSION="2.0 (SP5)"
ID="euleros"
ID_LIKE="rhel fedora centos"
VERSION_ID="2.0"
PRETTY_NAME="EulerOS 2.0 (SP5)"
ANSI_COLOR="0;31"

EulerOS is a self-developed operating system. Do you mean that I have to build the Arrow-java-dataset JNI code based on the OS image you provide?

kou commented 5 months ago

EulerOS is a self-developed operating system.

OK. I can't help you more because I can't use it.

Do you mean that I have to build the Arrow-java-dataset JNI code based on the OS image you provide?

No. I mean that you can refer them how to prepare your build environment.

jaynewchen commented 5 months ago

EulerOS is a self-developed operating system.

OK. I can't help you more because I can't use it.

Do you mean that I have to build the Arrow-java-dataset JNI code based on the OS image you provide?

No. I mean that you can refer them how to prepare your build environment.

thanks.