search

kwai / blaze

Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core.
Apache License 2.0
1.3k stars 121 forks source link

build libblaze.so failed in linux #593

Closed harveyyue closed 1 month ago

harveyyue commented 1 month ago

I try to build the project in linux or docker centos7 and always get the "could not compile gxhash (lib) due to 1 previous error" as below. The command cargo fix --all --allow-dirty --allow-staged --allow-no-vcs also need to set the proper value to the env variable RUSTFLAGS.

[INFO] --- exec-maven-plugin:3.0.0:exec (prepare-native-libs) @ assembly --- [INFO] [main] old build-checksum: /usr/bin/cat: ./.build-checksum_release-so.cache: No such file or directory [INFO] [main] new build-checksum: 9881c8434d0447c02ba47b1a9e0faa3c - [INFO] [main] Running cargo fix... [ERROR] [main] info: syncing channel updates for 'nightly-2024-06-27-x86_64-unknown-linux-gnu' [ERROR] [main] info: latest update on 2024-06-27, rust version 1.81.0-nightly (4bc39f028 2024-06-26) [ERROR] [main] info: downloading component 'cargo' [ERROR] [main] info: downloading component 'clippy' [ERROR] [main] info: downloading component 'rust-docs' [ERROR] [main] info: downloading component 'rust-src' [ERROR] [main] info: downloading component 'rust-std' [ERROR] [main] info: downloading component 'rustc' [ERROR] [main] info: downloading component 'rustfmt' [ERROR] [main] info: installing component 'cargo' [ERROR] [main] info: installing component 'clippy' [ERROR] [main] info: installing component 'rust-docs' [ERROR] [main] info: installing component 'rust-src' [ERROR] [main] info: installing component 'rust-std' [ERROR] [main] info: installing component 'rustc' [ERROR] [main] info: installing component 'rustfmt' [ERROR] [main] warning: virtual workspace defaulting to resolver = "1" despite one or more workspace members being on edition 2021 which implies resolver = "2" [ERROR] [main] note: to keep the current resolver, specify workspace.resolver = "1" in the workspace root's manifest [ERROR] [main] note: to use the edition 2021 resolver, specify workspace.resolver = "2" in the workspace root's manifest [ERROR] [main] note: for more details see https://doc.rust-lang.org/cargo/reference/resolver.html#resolver-versions [ERROR] [main] warning: virtual workspace defaulting to resolver = "1" despite one or more workspace members being on edition 2021 which implies resolver = "2" [ERROR] [main] note: to keep the current resolver, specify workspace.resolver = "1" in the workspace root's manifest [ERROR] [main] note: to use the edition 2021 resolver, specify workspace.resolver = "2" in the workspace root's manifest [ERROR] [main] note: for more details see https://doc.rust-lang.org/cargo/reference/resolver.html#resolver-versions [ERROR] [main] Compiling proc-macro2 v1.0.86 [ERROR] [main] Compiling unicode-ident v1.0.13 [ERROR] [main] Compiling libc v0.2.158 [ERROR] [main] Compiling autocfg v1.3.0 [ERROR] [main] Compiling cfg-if v1.0.0 [ERROR] [main] Compiling version_check v0.9.5 [ERROR] [main] Compiling byteorder v1.5.0 [ERROR] [main] Compiling libm v0.2.8 [ERROR] [main] Compiling quote v1.0.37 [ERROR] [main] Compiling getrandom v0.2.15 [ERROR] [main] Compiling crunchy v0.2.2 [ERROR] [main] Compiling syn v2.0.77 [ERROR] [main] Compiling num-traits v0.2.19 [ERROR] [main] Compiling tiny-keccak v2.0.2 [ERROR] [main] Compiling once_cell v1.20.0 [ERROR] [main] Compiling memchr v2.7.4 [ERROR] [main] Compiling const-random-macro v0.1.16 [ERROR] [main] Compiling aho-corasick v1.1.3 [ERROR] [main] Compiling jobserver v0.1.32 [ERROR] [main] Compiling regex-syntax v0.8.4 [ERROR] [main] Compiling shlex v1.3.0 [ERROR] [main] Compiling zerocopy-derive v0.7.35 [ERROR] [main] Compiling cc v1.1.18 [ERROR] [main] Compiling regex-automata v0.4.7 [ERROR] [main] Compiling ahash v0.8.11 [ERROR] [main] Compiling zerocopy v0.7.35 [ERROR] [main] Compiling rand_core v0.6.4 [ERROR] [main] Compiling siphasher v0.3.11 [ERROR] [main] Compiling ppv-lite86 v0.2.20 [ERROR] [main] Checking bytes v1.7.2 [ERROR] [main] Compiling phf_shared v0.11.2 [ERROR] [main] Compiling regex v1.10.6 [ERROR] [main] Compiling rand_chacha v0.3.1 [ERROR] [main] Compiling rand v0.8.5 [ERROR] [main] Compiling parse-zoneinfo v0.3.1 [ERROR] [main] Compiling phf v0.11.2 [ERROR] [main] Compiling phf_generator v0.11.2 [ERROR] [main] Compiling serde v1.0.210 [ERROR] [main] Compiling serde_derive v1.0.210 [ERROR] [main] Compiling phf_codegen v0.11.2 [ERROR] [main] Checking num-integer v0.1.46 [ERROR] [main] Compiling pkg-config v0.3.30 [ERROR] [main] Checking const-random v0.1.18 [ERROR] [main] Checking num-bigint v0.4.6 [ERROR] [main] Checking iana-time-zone v0.1.60 [ERROR] [main] Checking bitflags v2.6.0 [ERROR] [main] Checking allocator-api2 v0.2.18 [ERROR] [main] Checking hashbrown v0.14.5 [ERROR] [main] Checking num-rational v0.4.2 [ERROR] [main] Checking chrono v0.4.38 [ERROR] [main] Checking num-iter v0.1.45 [ERROR] [main] Compiling chrono-tz-build v0.2.1 [ERROR] [main] Checking num-complex v0.4.6 [ERROR] [main] Compiling heck v0.5.0 [ERROR] [main] Compiling chrono-tz v0.8.6 [ERROR] [main] Checking num v0.4.3 [ERROR] [main] Checking half v2.4.1 [ERROR] [main] Checking static_assertions v1.1.0 [ERROR] [main] Checking arrow-schema v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Compiling rustversion v1.0.17 [ERROR] [main] Checking arrow-buffer v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking lexical-util v0.8.5 [ERROR] [main] Checking arrow-data v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Compiling zstd-sys v2.0.13+zstd.1.5.6 [ERROR] [main] Compiling syn v1.0.109 [ERROR] [main] Compiling semver v1.0.23 [ERROR] [main] Checking pin-project-lite v0.2.14 [ERROR] [main] Compiling strum_macros v0.26.4 [ERROR] [main] Checking lexical-parse-integer v0.8.6 [ERROR] [main] Checking lexical-write-integer v0.8.5 [ERROR] [main] Checking futures-core v0.3.30 [ERROR] [main] Checking lexical-write-float v0.8.5 [ERROR] [main] Checking lexical-parse-float v0.8.5 [ERROR] [main] Checking strum v0.26.3 [ERROR] [main] Compiling slab v0.4.9 [ERROR] [main] Compiling lock_api v0.4.12 [ERROR] [main] Checking unicode-width v0.1.13 [ERROR] [main] Checking futures-sink v0.3.30 [ERROR] [main] Compiling parking_lot_core v0.9.10 [ERROR] [main] Checking arrow-array v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking comfy-table v7.1.1 [ERROR] [main] Compiling rustc_version v0.4.1 [ERROR] [main] Checking lexical-core v0.8.5 [ERROR] [main] Compiling rustix v0.38.37 [ERROR] [main] Checking arrow-select v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking smallvec v1.13.2 [ERROR] [main] Checking ryu v1.0.18 [ERROR] [main] Checking itoa v1.0.11 [ERROR] [main] Checking scopeguard v1.2.0 [ERROR] [main] Checking futures-io v0.3.30 [ERROR] [main] Checking tinyvec_macros v0.1.1 [ERROR] [main] Checking base64 v0.21.7 [ERROR] [main] Checking tinyvec v1.8.0 [ERROR] [main] Checking arrow-cast v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Compiling flatbuffers v23.5.26 [ERROR] [main] Checking futures-channel v0.3.30 [ERROR] [main] Compiling tokio-macros v2.4.0 [ERROR] [main] Compiling futures-macro v0.3.30 [ERROR] [main] Checking futures-task v0.3.30 [ERROR] [main] Compiling doc-comment v0.3.3 [ERROR] [main] Checking pin-utils v0.1.0 [ERROR] [main] Compiling zstd-safe v7.2.1 [ERROR] [main] Checking futures-util v0.3.30 [ERROR] [main] Checking tokio v1.40.0 [ERROR] [main] Checking unicode-normalization v0.1.23 [ERROR] [main] Checking twox-hash v1.6.3 [ERROR] [main] Checking alloc-no-stdlib v2.0.4 [ERROR] [main] Checking bitflags v1.3.2 [ERROR] [main] Checking unicode-bidi v0.3.15 [ERROR] [main] Checking log v0.4.22 [ERROR] [main] Checking equivalent v1.0.1 [ERROR] [main] Checking adler2 v2.0.0 [ERROR] [main] Checking either v1.13.0 [ERROR] [main] Compiling heck v0.4.1 [ERROR] [main] Compiling serde_json v1.0.96 (https://github.com/blaze-init/json?branch=v1.0.96-blaze#95fa6cba) [ERROR] [main] Checking percent-encoding v2.3.1 [ERROR] [main] Compiling snafu-derive v0.7.5 [ERROR] [main] Checking form_urlencoded v1.2.1 [ERROR] [main] Checking miniz_oxide v0.8.0 [ERROR] [main] Checking idna v0.5.0 [ERROR] [main] Checking indexmap v2.5.0 [ERROR] [main] Checking alloc-stdlib v0.2.2 [ERROR] [main] Checking lz4_flex v0.11.3 [ERROR] [main] Checking futures-executor v0.3.30 [ERROR] [main] Checking parking_lot v0.12.3 [ERROR] [main] Checking csv-core v0.1.11 [ERROR] [main] Checking tracing-core v0.1.32 [ERROR] [main] Compiling tracing-attributes v0.1.27 [ERROR] [main] Checking num_cpus v1.16.0 [ERROR] [main] Checking crc32fast v1.4.2 [ERROR] [main] Compiling snap v1.1.1 [ERROR] [main] Compiling paste v1.0.15 [ERROR] [main] Checking same-file v1.0.6 [ERROR] [main] Compiling typenum v1.17.0 [ERROR] [main] Checking walkdir v2.5.0 [ERROR] [main] Checking flate2 v1.0.33 [ERROR] [main] Checking tracing v0.1.40 [ERROR] [main] Checking threadpool v1.8.1 [ERROR] [main] Checking zstd v0.13.2 [ERROR] [main] Checking csv v1.3.0 [ERROR] [main] Checking snafu v0.7.5 [ERROR] [main] Checking arrow-ipc v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking futures v0.3.30 [ERROR] [main] Checking brotli-decompressor v2.5.1 [ERROR] [main] Checking url v2.5.2 [ERROR] [main] Checking itertools v0.12.1 [ERROR] [main] Compiling async-trait v0.1.82 [ERROR] [main] Checking ordered-float v2.10.1 [ERROR] [main] Compiling generic-array v0.14.7 [ERROR] [main] Checking lazy_static v1.5.0 [ERROR] [main] Checking integer-encoding v3.0.4 [ERROR] [main] Checking humantime v2.1.0 [ERROR] [main] Checking thrift v0.17.0 [ERROR] [main] Checking arrow-csv v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking object_store v0.9.1 [ERROR] [main] Checking arrow-json v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking brotli v3.5.0 [ERROR] [main] Checking arrow-string v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking arrow-ord v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking arrow-row v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking arrow-arith v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Compiling sqlparser_derive v0.2.2 [ERROR] [main] Compiling seq-macro v0.3.5 [ERROR] [main] Checking arrow v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking sqlparser v0.43.1 [ERROR] [main] Checking parquet v50.0.0 (https://github.com/blaze-init/arrow-rs.git?rev=7471d70f7ae6edd5d4da82b7d966a8ede720e499#7471d70f) [ERROR] [main] Checking linux-raw-sys v0.4.14 [ERROR] [main] Checking block-buffer v0.10.4 [ERROR] [main] Checking crypto-common v0.1.6 [ERROR] [main] Checking fastrand v2.1.1 [ERROR] [main] Checking subtle v2.6.1 [ERROR] [main] Checking digest v0.10.7 [ERROR] [main] Checking datafusion-common v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Checking tempfile v3.12.0 [ERROR] [main] Checking dashmap v5.5.3 [ERROR] [main] Compiling blake3 v1.5.4 [ERROR] [main] Compiling lzma-sys v0.1.20 [ERROR] [main] Compiling bzip2-sys v0.1.11+1.0.8 [ERROR] [main] Checking arrayvec v0.7.6 [ERROR] [main] Checking datafusion-expr v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Checking arrayref v0.3.9 [ERROR] [main] Checking hex v0.4.3 [ERROR] [main] Checking constant_time_eq v0.3.1 [ERROR] [main] Checking cpufeatures v0.2.14 [ERROR] [main] Compiling thiserror v1.0.63 [ERROR] [main] Checking fixedbitset v0.4.2 [ERROR] [main] Checking petgraph v0.6.5 [ERROR] [main] Checking sha2 v0.10.8 [ERROR] [main] Checking datafusion-execution v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Checking blake2 v0.10.6 [ERROR] [main] Checking md-5 v0.10.6 [ERROR] [main] Checking uuid v1.10.0 [ERROR] [main] Compiling thiserror-impl v1.0.63 [ERROR] [main] Compiling anyhow v1.0.88 [ERROR] [main] Checking unicode-segmentation v1.12.0 [ERROR] [main] Checking datafusion-physical-expr v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Checking bzip2 v0.4.4 [ERROR] [main] Compiling jni v0.20.0 [ERROR] [main] Checking datafusion-functions v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Checking datafusion-functions-array v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Checking datafusion-sql v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Checking tokio-util v0.7.12 [ERROR] [main] Checking glob v0.3.1 [ERROR] [main] Checking combine v4.6.7 [ERROR] [main] Checking datafusion-optimizer v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Checking datafusion-physical-plan v36.0.0 (https://github.com/harveyyue/datafusion.git?rev=d33877f8fbc7c57de946dc6081b2b357eedd0df9#d33877f8) [ERROR] [main] Compiling radium v0.7.0 [ERROR] [main] Checking cesu8 v1.1.0 [ERROR] [main] Checking jni-sys v0.3.0 [ERROR] [main] Compiling ptr_meta_derive v0.2.0 [ERROR] [main] Compiling bigdecimal v0.4.5 [ERROR] [main] Checking tap v1.0.1 [ERROR] [main] Checking wyz v0.5.1 [ERROR] [main] Checking ptr_meta v0.2.0 [ERROR] [main] Checking funty v2.0.0 [ERROR] [main] Checking slimmer_box v0.6.5 [ERROR] [main] Compiling itertools v0.13.0 [ERROR] [main] Checking bitvec v1.0.1 [ERROR] [main] Checking unchecked-index v0.2.2 [ERROR] [main] Checking radsort v0.1.1 [ERROR] [main] Compiling prost-derive v0.13.3 [ERROR] [main] Compiling chrono-tz-build v0.3.0 [ERROR] [main] Checking simdutf8 v0.1.4 [ERROR] [main] Compiling prettyplease v0.2.22 [ERROR] [main] Compiling zstd-safe v6.0.6 [ERROR] [main] Compiling prost v0.13.3 [ERROR] [main] Checking faststr v0.2.23 [ERROR] [main] Compiling chrono-tz v0.9.0 [ERROR] [main] Checking bumpalo v3.16.0 [ERROR] [main] Compiling prost-derive v0.12.6 [ERROR] [main] Checking sonic-rs v0.3.13 [ERROR] [main] Compiling prost-types v0.13.2 [ERROR] [main] Compiling snafu-derive v0.8.4 [ERROR] [main] Compiling multimap v0.10.0 [ERROR] [main] Compiling gxhash v3.4.1 [ERROR] [main] Checking snafu v0.8.4 [ERROR] [main] Compiling prost-build v0.13.2 [ERROR] [main] Checking zstd v0.12.4 [ERROR] [main] Checking prost v0.12.6 [ERROR] [main] Checking lzokay-native v0.1.0 [ERROR] [main] Checking fallible-streaming-iterator v0.1.9 [ERROR] [main] Compiling tonic-build v0.12.2 [ERROR] [main] error: Gxhash requires aes and sse2 intrinsics. Make sure the processor supports it and build with RUSTFLAGS="-C target-cpu=native" or RUSTFLAGS="-C target-feature=+aes,+sse2". [ERROR] [main] --> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/gxhash-3.4.1/src/gxhash/platform/x86.rs:2:1 [ERROR] [main] | [ERROR] [main] 2 | compile_error!{"Gxhash requires aes and sse2 intrinsics. Make sure the processor supports it and build with RUSTFLAGS=\"-C target-cpu=native\" or RUSTFLAGS=\"-C target-feature=+aes,+sse2\"."} [ERROR] [main] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [ERROR] [main] [ERROR] [main] error: could not compile gxhash (lib) due to 1 previous error [ERROR] [main] warning: build failed, waiting for other jobs to finish... [ERROR] Command execution failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 101 (Exit value: 101) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166) at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:982) at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:929) at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:452) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345) at org.apache.maven.cli.MavenCli.main(MavenCli.java:191) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)