search

delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python
https://delta-io.github.io/delta-rs/
Apache License 2.0
2.28k stars 403 forks source link

0.6.1 fails to install on MacOS Monterey #808

Closed ahlag closed 2 years ago

ahlag commented 2 years ago

Environment

Delta-rs version: 0.6.1

Binding: python

Environment:

Bug

❯ pip install deltalake                                                                                                                             ─╯
Collecting deltalake
  Using cached deltalake-0.6.1.tar.gz (4.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pyarrow>=7
  Using cached pyarrow-9.0.0-cp38-cp38-macosx_10_13_x86_64.whl (24.0 MB)
Collecting numpy>=1.16.6
  Using cached numpy-1.23.3-cp38-cp38-macosx_10_9_x86_64.whl (18.1 MB)
Building wheels for collected packages: deltalake
  Building wheel for deltalake (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for deltalake (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [266 lines of output]
      Running `maturin pep517 build-wheel -i /Users/changyonglik/Desktop/opensource/tmp/feathr/.venvs/feathr-dev/bin/python --compatibility off`
         Compiling libc v0.2.132
         Compiling autocfg v1.1.0
         Compiling proc-macro2 v1.0.43
         Compiling unicode-ident v1.0.4
         Compiling quote v1.0.21
         Compiling syn v1.0.99
         Compiling cfg-if v1.0.0
         Compiling memchr v2.5.0
         Compiling log v0.4.17
         Compiling version_check v0.9.4
         Compiling serde_derive v1.0.144
         Compiling serde v1.0.144
         Compiling core-foundation-sys v0.8.3
         Compiling smallvec v1.9.0
         Compiling parking_lot_core v0.9.3
         Compiling pin-project-lite v0.2.9
         Compiling scopeguard v1.1.0
         Compiling once_cell v1.14.0
         Compiling typenum v1.15.0
         Compiling futures-core v0.3.24
         Compiling bytes v1.2.1
         Compiling itoa v1.0.3
         Compiling futures-sink v0.3.24
         Compiling futures-task v0.3.24
         Compiling futures-channel v0.3.24
         Compiling futures-util v0.3.24
         Compiling bitflags v1.3.2
         Compiling pin-utils v0.1.0
         Compiling futures-io v0.3.24
         Compiling hashbrown v0.12.3
         Compiling fnv v1.0.7
         Compiling httparse v1.8.0
         Compiling untrusted v0.7.1
         Compiling lazy_static v1.4.0
         Compiling ryu v1.0.11
         Compiling spin v0.5.2
         Compiling serde_json v1.0.85
         Compiling try-lock v0.2.3
         Compiling tower-service v0.3.2
         Compiling semver v1.0.13
         Compiling httpdate v1.0.2
         Compiling base64 v0.13.0
         Compiling subtle v2.4.1
         Compiling rustls v0.20.6
         Compiling native-tls v0.2.10
         Compiling remove_dir_all v0.5.3
         Compiling percent-encoding v2.2.0
         Compiling fastrand v1.8.0
         Compiling async-trait v0.1.57
         Compiling cpufeatures v0.2.5
         Compiling target-lexicon v0.12.4
         Compiling thiserror v1.0.35
         Compiling crc32fast v1.3.2
         Compiling opaque-debug v0.3.0
         Compiling tinyvec_macros v0.1.0
         Compiling xml-rs v0.8.4
         Compiling hex v0.4.3
         Compiling shlex v1.1.0
         Compiling zeroize v1.5.7
         Compiling static_assertions v1.1.0
         Compiling unicode-bidi v0.3.8
         Compiling encoding_rs v0.8.31
         Compiling ppv-lite86 v0.2.16
         Compiling mime v0.3.16
         Compiling ipnet v2.5.0
         Compiling regex-syntax v0.6.27
         Compiling zstd-safe v5.0.2+zstd.1.5.2
         Compiling alloc-no-stdlib v2.0.4
         Compiling doc-comment v0.3.3
         Compiling dyn-clone v1.0.9
         Compiling regex-automata v0.1.10
         Compiling heck v0.4.0
         Compiling snap v1.0.5
         Compiling integer-encoding v1.1.7
         Compiling adler v1.0.2
         Compiling unindent v0.1.10
         Compiling indoc v1.0.7
         Compiling itoa v0.4.8
         Compiling byteorder v1.4.3
         Compiling RustyXML v0.3.0
         Compiling md5 v0.7.0
         Compiling parquet v20.0.0
         Compiling half v2.1.0
         Compiling either v1.8.0
         Compiling same-file v1.0.6
         Compiling seq-macro v0.3.1
         Compiling maplit v1.0.2
         Compiling termcolor v1.1.3
         Compiling humantime v2.1.0
         Compiling tracing-core v0.1.29
         Compiling generic-array v0.14.6
         Compiling ahash v0.7.6
         Compiling lock_api v0.4.8
         Compiling num-traits v0.2.15
         Compiling slab v0.4.7
         Compiling tokio v1.21.1
         Compiling indexmap v1.9.1
         Compiling num-integer v0.1.45
         Compiling num-bigint v0.4.3
         Compiling num-rational v0.4.1
         Compiling num-iter v0.1.43
         Compiling http v0.2.8
         Compiling form_urlencoded v1.1.0
         Compiling tinyvec v1.6.0
         Compiling rustls-pemfile v1.0.1
         Compiling lexical-util v0.8.5
         Compiling alloc-stdlib v0.2.2
         Compiling miniz_oxide v0.5.4
         Compiling walkdir v2.3.2
         Compiling itertools v0.10.4
         Compiling http-body v0.4.5
         Compiling lexical-write-integer v0.8.5
         Compiling lexical-parse-integer v0.8.6
         Compiling unicode-normalization v0.1.21
         Compiling brotli-decompressor v2.3.2
         Compiling want v0.3.0
         Compiling iana-time-zone v0.1.48
         Compiling aho-corasick v0.7.19
         Compiling csv-core v0.1.10
         Compiling jobserver v0.1.24
         Compiling num_cpus v1.13.1
         Compiling mio v0.8.4
         Compiling signal-hook-registry v1.4.0
         Compiling socket2 v0.4.7
         Compiling core-foundation v0.9.3
         Compiling security-framework-sys v2.6.1
         Compiling time v0.1.44
         Compiling tempfile v3.3.0
         Compiling getrandom v0.2.7
         Compiling dirs-sys-next v0.1.2
         Compiling atty v0.2.14
         Compiling errno v0.2.8
         Compiling lexical-write-float v0.8.5
         Compiling lexical-parse-float v0.8.5
         Compiling idna v0.3.0
         Compiling brotli v3.3.4
         Compiling rustc_version v0.4.0
         Compiling flate2 v1.0.24
         Compiling regex v1.6.0
         Compiling cc v1.0.73
         Compiling pyo3-build-config v0.16.6
         Compiling threadpool v1.8.1
         Compiling security-framework v2.7.0
         Compiling rand_core v0.6.3
         Compiling dirs-next v2.0.0
         Compiling parking_lot v0.12.1
         Compiling lexical-core v0.8.5
         Compiling rusoto_core v0.48.0
         Compiling azure_core v0.2.2
         Compiling rand_chacha v0.3.1
         Compiling digest v0.9.0
         Compiling block-buffer v0.9.0
         Compiling crypto-mac v0.11.1
         Compiling crypto-common v0.1.6
         Compiling block-buffer v0.10.3
         Compiling env_logger v0.9.0
         Compiling rustls-native-certs v0.6.2
         Compiling num-complex v0.4.2
         Compiling ordered-float v1.1.1
         Compiling glibc_version v0.1.2
         Compiling rand v0.8.5
         Compiling ring v0.16.20
         Compiling zstd-sys v2.0.1+zstd.1.5.2
         Compiling lz4-sys v1.9.4
         Compiling md-5 v0.9.1
         Compiling sha2 v0.9.9
         Compiling hmac v0.11.0
         Compiling digest v0.10.3
         Compiling thrift v0.13.0
         Compiling sha2 v0.10.5
         Compiling hmac v0.12.1
         Compiling parquet-format v4.0.0
         Compiling deltalake v0.4.1 (/private/var/folders/8k/ph3xqvb11k32hjl7r78jhzh00000gn/T/pip-install-a0rdft3u/deltalake_9d32a96beba5408893bf86a635ac980f/local_dependencies/deltalake)
         Compiling pyo3-ffi v0.16.6
         Compiling pyo3 v0.16.6
         Compiling num v0.4.0
         Compiling pyo3-macros-backend v0.16.6
         Compiling tokio-macros v1.8.0
         Compiling futures-macro v0.3.24
         Compiling tracing-attributes v0.1.22
         Compiling thiserror-impl v1.0.35
         Compiling pin-project-internal v1.0.12
         Compiling multiversion-macros v0.6.1
         Compiling snafu-derive v0.7.1
         Compiling pyo3-macros v0.16.6
         Compiling tracing v0.1.36
         Compiling lz4 v1.24.0
         Compiling pin-project v1.0.12
         Compiling multiversion v0.6.1
         Compiling webpki v0.22.0
         Compiling sct v0.7.0
         Compiling flatbuffers v2.1.2
         Compiling webpki-roots v0.22.4
         Compiling snafu v0.7.1
         Compiling futures-executor v0.3.24
         Compiling futures v0.3.24
         Compiling tokio-util v0.7.4
         Compiling tokio-rustls v0.23.4
         Compiling tokio-native-tls v0.3.0
         Compiling h2 v0.3.14
         Compiling chrono v0.4.22
         Compiling serde_urlencoded v0.7.1
         Compiling url v2.3.1
         Compiling serde_path_to_error v0.1.8
         Compiling uuid v1.1.2
         Compiling serde-xml-rs v0.5.1
         Compiling bstr v0.2.17
         Compiling uuid v0.8.2
         Compiling quick-xml v0.23.1
         Compiling csv v1.1.6
         Compiling hyper v0.14.20
         Compiling zstd v0.11.2+zstd.1.5.2
         Compiling arrow v20.0.0
         Compiling hyper-rustls v0.23.0
         Compiling hyper-tls v0.5.0
         Compiling rusoto_credential v0.48.0
         Compiling reqwest v0.11.11
         Compiling rusoto_signature v0.48.0
         Compiling oauth2 v4.2.3
      error: could not compile `rusoto_core` due to 4 previous errors
      warning: build failed, waiting for other jobs to finish...
      💥 maturin failed
        Caused by: Failed to build a native library through cargo
        Caused by: Cargo build finished with "exit status: 101": `"cargo" "rustc" "--release" "--message-format" "json" "--lib" "--" "-C" "link-arg=-undefined" "-C" "link-arg=dynamic_lookup" "-C" "link-args=-Wl,-install_name,@rpath/deltalake.cpython-38-darwin.so"`
      🍹 Building a mixed python/rust project
      🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
      🐍 Not using a specific python interpreter
      error[E0252]: the name `tls` is defined multiple times
        --> /Users/changyonglik/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_core-0.48.0/src/lib.rs:25:5
         |
      23 | use hyper_rustls as tls;
         |     ------------------- previous import of the module `tls` here
      24 | #[cfg(feature = "native-tls")]
      25 | use hyper_tls as tls;
         |     ^^^^^^^^^^^^^^^^ `tls` reimported here
         |
         = note: `tls` must be defined only once in the type namespace of this module
      help: you can use `as` to change the binding name of the import
         |
      25 | use hyper_tls as other_tls;
         |     ~~~~~~~~~~~~~~~~~~~~~~

      error[E0599]: no function or associated item named `new` found for struct `hyper_rustls::HttpsConnector` in the current scope
         --> /Users/changyonglik/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_core-0.48.0/src/request.rs:224:41
          |
      224 |         let connector = HttpsConnector::new();
          |                                         ^^^ function or associated item not found in `hyper_rustls::HttpsConnector<_>`

      error[E0599]: no function or associated item named `new` found for struct `hyper_rustls::HttpsConnector` in the current scope
         --> /Users/changyonglik/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_core-0.48.0/src/request.rs:244:41
          |
      244 |         let connector = HttpsConnector::new();
          |                                         ^^^ function or associated item not found in `hyper_rustls::HttpsConnector<_>`

      error: aborting due to 3 previous errors

      Some errors have detailed explanations: E0252, E0599.

      For more information about an error, try `rustc --explain E0252`.

      Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/Users/changyonglik/Desktop/opensource/tmp/feathr/.venvs/feathr-dev/bin/python', '--compatibility', 'off'] returned non-zero exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for deltalake
Failed to build deltalake
ERROR: Could not build wheels for deltalake, which is required to install pyproject.toml-based projects

What happened:

What you expected to happen:

How to reproduce it:

More details:

fvaleye commented 2 years ago

Hello @ahlag,

Thank you for your report.

Sorry about that. Indeed, the x86_64 wheel for Mac is unavailable in Pypy for the 0.6.1 version. Anyway, it doesn't explain why it fails to install deltalake from the archive .tar.gz.

810 will temporarily fix the issue while we need more time before knowing more about the root cause.

ahlag commented 2 years ago

Hi @fvaleye

Thank you for creating a quick fix for the issue. What version is available for MacOS as of now? We need to fix the deltalake version in requirements.txt.

fvaleye commented 2 years ago

Could you try the 0.5.7 version to see how it goes?

ahlag commented 2 years ago

Hi @fvaleye

I have tried pip install deltalake but it still didn't work on MacOS. Is this issue fixed or should I move on to deltalake 0.5.7?

fvaleye commented 2 years ago

Hello @ahlag 👋,

The new version has not been released in PyPi yet (maybe 0.6.2 version). In the meantime, you can use 0.5.7, and I will keep you informed when the new version is out.

ahlag commented 2 years ago

Thanks!

fvaleye commented 2 years ago

Hello @ahlag,

The release 0.6.2 has been done. Could you please test this version on your side and see how it goes?

Thanks!

ahlag commented 2 years ago

@fvaleye Thank you so much! It worked!

fvaleye commented 2 years ago

Thanks for testing!