search

pola-rs / polars

Dataframes powered by a multithreaded, vectorized query engine, written in Rust
https://docs.pola.rs
Other
29.2k stars 1.84k forks source link

Build fail #18267

Open kalimalrazif opened 3 weeks ago

kalimalrazif commented 3 weeks ago

Checks

Reproducible example

Log output

error[E0433]: failed to resolve: use of undeclared type `ParquetAsyncReader`
   --> /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-plan-0.42.0/src/plans/functions/count.rs:128:17
    |
128 |                 ParquetAsyncReader::from_uri(&path.to_string_lossy(), cloud_options, None).await?;
    |                 ^^^^^^^^^^^^^^^^^^ use of undeclared type `ParquetAsyncReader`
    |
help: a struct with a similar name exists
    |
128 |                 ParquetReader::from_uri(&path.to_string_lossy(), cloud_options, None).await?;
    |                 ~~~~~~~~~~~~~
help: consider importing this struct
    |
3   + use polars_io::prelude::ParquetAsyncReader;
    |

Issue description

I try to run: "cargo build"

Expected behavior

Error-free compilation process

Installed versions

polars-core = "0.42.0" polars-lazy = { version = "0.42.0", features = ["async", "dtype-full", "fmt", "parquet", "regex", "strings", "timezones", "tokio"] } polars-io = { version = "0.42.0", features = ["async", "polars-parquet", "parquet"] }
cmdlineluser commented 3 weeks ago

Seems to be the same issue as https://github.com/pola-rs/polars/issues/18258

Both of these are gated by the cloud feature - I'm guessing this is new?

https://github.com/pola-rs/polars/blob/c88da1f6266a4f7f85d1487485e5943b84a2ac7f/crates/polars-io/src/parquet/read/reader.rs#L252-L253

kalimalrazif commented 3 weeks ago

I downgraded the version to 0.41.3 and I have the same result

error[E0433]: failed to resolve: use of undeclared type `ParquetAsyncReader`
   --> /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-plan-0.41.3/src/plans/functions/count.rs:125:17
    |
125 |                 ParquetAsyncReader::from_uri(&path.to_string_lossy(), cloud_options, None).await?;
    |                 ^^^^^^^^^^^^^^^^^^ use of undeclared type `ParquetAsyncReader`
    |
help: a struct with a similar name exists
    |
125 |                 ParquetReader::from_uri(&path.to_string_lossy(), cloud_options, None).await?;
    |                 ~~~~~~~~~~~~~
help: consider importing this struct
    |
3   + use polars_io::prelude::ParquetAsyncReader;
    |

But despite including in my Cargo.toml:

polars-core = "0.41.3"
polars-lazy = { version = "0.41.3", features = ["async", "dtype-full", "fmt", "parquet", "regex", "strings", "timezones", "tokio"] }
polars-io = { version = "0.41.3", features = ["parquet"] }

The building process shows this:

   Compiling polars-row v0.35.4
   Compiling polars-row v0.41.3
   Compiling polars-parquet v0.41.3
   Compiling polars-time v0.41.3
   Compiling polars-io v0.41.3
   Compiling polars-time v0.35.4
   Compiling polars-io v0.35.4
kalimalrazif commented 3 weeks ago

Hi, removing some features mi code builds again, the new Cargo.toml is like this:

polars-core = "0.41.3"
polars-lazy = { version = "0.41.3", features = ["dtype-full", "fmt", "parquet", "regex", "strings"] }
kalimalrazif commented 3 weeks ago

Running the code got this error message:

thread 'main' panicked at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-lazy-0.41.3/src/scan/file_list_reader.rs:169:9:
Feature `async` must be enabled to use globbing patterns with cloud urls.
stack backtrace:
   0:     0x55f4a9325f2d - std::backtrace_rs::backtrace::libunwind::trace::h06bfed5b6683132f
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/../../backtrace/src/backtrace/libunwind.rs:116:5
   1:     0x55f4a9325f2d - std::backtrace_rs::backtrace::trace_unsynchronized::h30e0892c1e23c613
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55f4a9325f2d - std::sys::backtrace::_print_fmt::h702ddc0c19466cc1
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/sys/backtrace.rs:66:9
   3:     0x55f4a9325f2d - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h7b9fa4a70e398292
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/sys/backtrace.rs:39:26
   4:     0x55f4a93515db - core::fmt::rt::Argument::fmt::h1eab2d243de03c0e
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/core/src/fmt/rt.rs:173:76
   5:     0x55f4a93515db - core::fmt::write::hb46a6aeaf4aaf4a0
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/core/src/fmt/mod.rs:1178:21
   6:     0x55f4a93217e3 - std::io::Write::write_fmt::h7944ee05c08b6149
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/io/mod.rs:1823:15
   7:     0x55f4a9327222 - std::sys::backtrace::BacktraceLock::print::hac09b5d368b344ba
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/sys/backtrace.rs:42:9
   8:     0x55f4a9327222 - std::panicking::default_hook::{{closure}}::hfb3a7e9a6232257a
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:266:22
   9:     0x55f4a9326e8e - std::panicking::default_hook::h0b246b0a47fbfcd7
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:293:9
  10:     0x55f4a9327b1f - std::panicking::rust_panic_with_hook::hed2a518785b28232
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:797:13
  11:     0x55f4a93277d3 - std::panicking::begin_panic_handler::{{closure}}::h6f9b66bffc56b529
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:664:13
  12:     0x55f4a9326419 - std::sys::backtrace::__rust_end_short_backtrace::h858cdf2edfa45205
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/sys/backtrace.rs:170:18
  13:     0x55f4a9327494 - rust_begin_unwind
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:662:5
  14:     0x55f4a934e113 - core::panicking::panic_fmt::h7889063f14181452
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/core/src/panicking.rs:74:14
  15:     0x55f4a2da5ca0 - polars_lazy::scan::file_list_reader::expand_paths::hb6b888acc4b5e5bf
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-lazy-0.41.3/src/scan/file_list_reader.rs:169:9
  16:     0x55f4a2da6367 - polars_lazy::scan::file_list_reader::LazyFileListReader::expand_paths::h11bfeafe96d5f253
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-lazy-0.41.3/src/scan/file_list_reader.rs:339:9
  17:     0x55f4a2da6612 - <polars_lazy::scan::parquet::LazyParquetReader as polars_lazy::scan::file_list_reader::LazyFileListReader>::finish::h044535d377601b2d
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-lazy-0.41.3/src/scan/parquet.rs:63:13
  18:     0x55f4a2b594d1 - polars_lazy::scan::parquet::<impl polars_lazy::frame::LazyFrame>::scan_parquet::hcbc9a793e004a14d
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-lazy-0.41.3/src/scan/parquet.rs:154:9
  19:     0x55f4a2b85337 - fast_client::lookups::ip::IPLookup::new::hbe032c84717f6b28
                               at /home/nomar/Documents/Trabajos/EmQu/Development/fast_collector_client/src/lookups/ip/mod.rs:31:25
  20:     0x55f4a2b50228 - fast_client::run_client::{{closure}}::h9195d4ecb6355cbf
                               at /home/nomar/Documents/Trabajos/EmQu/Development/fast_collector_client/src/main.rs:146:21
  21:     0x55f4a2b4fc6e - fast_client::main::{{closure}}::h6c28704cb8438bd8
                               at /home/nomar/Documents/Trabajos/EmQu/Development/fast_collector_client/src/main.rs:122:33
  22:     0x55f4a2b4f65d - <core::pin::Pin<P> as core::future::future::Future>::poll::hf2811f21b9f02b0a
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/core/src/future/future.rs:123:9
  23:     0x55f4a2bba9ef - tokio::runtime::park::CachedParkThread::block_on::{{closure}}::hd158f0afce20fbb8
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/park.rs:281:63
  24:     0x55f4a2bba412 - tokio::runtime::coop::with_budget::h00cb79a36bc0fd6f
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/coop.rs:107:5
  25:     0x55f4a2bba412 - tokio::runtime::coop::budget::h62c537b0c690df9f
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/coop.rs:73:5
  26:     0x55f4a2bba412 - tokio::runtime::park::CachedParkThread::block_on::h7f2d3ab926ea9f3b
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/park.rs:281:31
  27:     0x55f4a2b0cfc0 - tokio::runtime::context::blocking::BlockingRegionGuard::block_on::h2eb16f419d157444
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/context/blocking.rs:66:9
  28:     0x55f4a2b10b0e - tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}::hb496fd8a036e6841
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/scheduler/multi_thread/mod.rs:87:13
  29:     0x55f4a2bc41c0 - tokio::runtime::context::runtime::enter_runtime::ha2cdb15c173dd216
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/context/runtime.rs:65:16
  30:     0x55f4a2b109f1 - tokio::runtime::scheduler::multi_thread::MultiThread::block_on::h519540d09656c259
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/scheduler/multi_thread/mod.rs:86:9
  31:     0x55f4a2c38292 - tokio::runtime::runtime::Runtime::block_on_inner::h62027ac4cb4e7490
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/runtime.rs:363:45
  32:     0x55f4a2c38649 - tokio::runtime::runtime::Runtime::block_on::h652248a37a080853
                               at /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.3/src/runtime/runtime.rs:333:13
  33:     0x55f4a2bc0602 - fast_client::main::h6aa99b3006666c79
                               at /home/nomar/Documents/Trabajos/EmQu/Development/fast_collector_client/src/main.rs:121:13
  34:     0x55f4a2b8f7ab - core::ops::function::FnOnce::call_once::h1dc9da905f046007
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/core/src/ops/function.rs:250:5
  35:     0x55f4a2b80d3e - std::sys::backtrace::__rust_begin_short_backtrace::he55c9c0ac0b551d0
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/sys/backtrace.rs:154:18
  36:     0x55f4a2b820c1 - std::rt::lang_start::{{closure}}::hefdc9b4a32dac173
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/rt.rs:164:18
  37:     0x55f4a9318d00 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::hd88bfc3d1486d5a3
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/core/src/ops/function.rs:284:13
  38:     0x55f4a9318d00 - std::panicking::try::do_call::ha8563cbd5fad7f25
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:554:40
  39:     0x55f4a9318d00 - std::panicking::try::h37721c1f06963490
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:518:19
  40:     0x55f4a9318d00 - std::panic::catch_unwind::hb24a87ea24e5f253
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panic.rs:345:14
  41:     0x55f4a9318d00 - std::rt::lang_start_internal::{{closure}}::he5ab27d1c79488cb
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/rt.rs:143:48
  42:     0x55f4a9318d00 - std::panicking::try::do_call::hbf057740e5ec7aa8
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:554:40
  43:     0x55f4a9318d00 - std::panicking::try::hcb7d4b1b43dc127a
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panicking.rs:518:19
  44:     0x55f4a9318d00 - std::panic::catch_unwind::h98658d6fda136867
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/panic.rs:345:14
  45:     0x55f4a9318d00 - std::rt::lang_start_internal::h3b3ddc082fe46b33
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/rt.rs:143:20
  46:     0x55f4a2b8209a - std::rt::lang_start::he070730392278120
                               at /rustc/91376f416222a238227c84a848d168835ede2cc3/library/std/src/rt.rs:163:17
  47:     0x55f4a2bc1b2e - main
  48:     0x7fbae45b3083 - __libc_start_main
                               at /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16
  49:     0x55f4a2ae956e - _start
  50:                0x0 - <unknown>

Process finished with exit code 101
kalimalrazif commented 3 weeks ago

Enabling async feature this is the build output:

❯ cargo build
warning: unused manifest key: build
    Blocking waiting for file lock on package cache
     Locking 12 packages to latest compatible versions
      Adding arrayref v0.3.8
      Adding blake3 v1.5.3
      Adding constant_time_eq v0.3.0
      Adding doc-comment v0.3.3
      Adding fs4 v0.8.4 (latest: v0.9.1)
      Adding itertools v0.13.0
      Adding object_store v0.10.2 (latest: v0.11.0)
      Adding same-file v1.0.6
      Adding snafu v0.7.5 (latest: v0.8.4)
      Adding snafu-derive v0.7.5 (latest: v0.8.4)
      Adding walkdir v2.5.0
      Adding winapi-util v0.1.9
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
   Compiling serde v1.0.208
   Compiling futures-macro v0.3.30
   Compiling futures-channel v0.3.30
   Compiling futures-io v0.3.30
   Compiling futures-task v0.3.30
   Compiling smartstring v1.0.1
   Compiling doc-comment v0.3.3
   Compiling snafu-derive v0.7.5
   Compiling async-trait v0.1.81
   Compiling same-file v1.0.6
   Compiling itertools v0.13.0
   Compiling serde_json v1.0.125
   Compiling rustix v0.38.34
   Compiling polars-arrow v0.41.3
   Compiling linux-raw-sys v0.4.14
   Compiling tokio-util v0.7.11
   Compiling polars-core v0.41.3
   Compiling walkdir v2.5.0
   Compiling blake3 v1.5.3
   Compiling async-stream-impl v0.3.5
   Compiling arrayref v0.3.8
   Compiling constant_time_eq v0.3.0
   Compiling polars-plan v0.41.3
   Compiling polars-lazy v0.41.3
   Compiling async-stream v0.3.5
   Compiling futures-util v0.3.30
   Compiling snafu v0.7.5
   Compiling futures-executor v0.3.30
   Compiling tower v0.4.13
   Compiling http-body-util v0.1.2
   Compiling futures v0.3.30
   Compiling parquet-format-safe v0.2.4
   Compiling hashbrown v0.14.5
   Compiling bitflags v2.6.0
   Compiling chrono v0.4.38
   Compiling polars-arrow-format v0.1.0
   Compiling log v0.4.22
   Compiling serde_urlencoded v0.7.1
   Compiling rust_decimal v1.36.0
   Compiling serde-value v0.7.0
   Compiling serde_with v1.14.0
   Compiling calamine v0.22.1
   Compiling crossterm v0.27.0
   Compiling raw-cpuid v11.1.0
   Compiling arrow-schema v52.2.0
   Compiling openssl v0.10.66
   Compiling comfy-table v7.1.1
   Compiling arrow-data v52.2.0
   Compiling byte-unit v5.1.4
   Compiling object_store v0.10.2
   Compiling now v0.1.3
   Compiling gelf v0.5.0
   Compiling chrono-tz v0.9.0
   Compiling indexmap v2.4.0
   Compiling arrow-array v52.2.0
   Compiling hashlink v0.8.4
   Compiling xattr v1.3.1
   Compiling fs4 v0.8.4
   Compiling tar v0.4.41
   Compiling native-tls v0.2.12
   Compiling kafka v0.10.0
   Compiling polars-utils v0.35.4
   Compiling cynic-parser v0.4.5
   Compiling h2 v0.4.5
   Compiling serde_yaml v0.9.34+deprecated
   Compiling polars-error v0.41.3
   Compiling tokio-native-tls v0.3.1
   Compiling libduckdb-sys v1.0.0
   Compiling polars-arrow v0.35.4
   Compiling polars-utils v0.41.3
   Compiling log4rs v1.3.0
   Compiling confy v0.6.1
   Compiling arrow-select v52.2.0
   Compiling arrow-row v52.2.0
   Compiling arrow-arith v52.2.0
   Compiling cynic-codegen v3.7.3
   Compiling hyper v1.4.1
   Compiling arrow-ord v52.2.0
   Compiling arrow-string v52.2.0
   Compiling arrow-cast v52.2.0
   Compiling hyper-util v0.1.7
   Compiling arrow v52.2.0
   Compiling hyper-tls v0.6.0
   Compiling reqwest v0.12.5
   Compiling fast-client v0.3.0 (/home/nomar/Documents/Trabajos/EmQu/Development/fast_collector_client)
   Compiling cynic-proc-macros v3.7.3
   Compiling cynic v3.7.3
   Compiling polars-compute v0.41.3
   Compiling polars-row v0.41.3
   Compiling polars-row v0.35.4
   Compiling polars-parquet v0.41.3
   Compiling polars-core v0.35.4
   Compiling polars-ops v0.41.3
   Compiling polars-io v0.41.3
   Compiling polars-ops v0.35.4
   Compiling polars-time v0.41.3
   Compiling polars-time v0.35.4
   Compiling polars-io v0.35.4
   Compiling polars v0.35.4
error[E0433]: failed to resolve: use of undeclared type `ParquetAsyncReader`
   --> /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-plan-0.41.3/src/plans/functions/count.rs:125:17
    |
125 |                 ParquetAsyncReader::from_uri(&path.to_string_lossy(), cloud_options, None).await?;
    |                 ^^^^^^^^^^^^^^^^^^ use of undeclared type `ParquetAsyncReader`
    |
help: a struct with a similar name exists
    |
125 |                 ParquetReader::from_uri(&path.to_string_lossy(), cloud_options, None).await?;
    |                 ~~~~~~~~~~~~~
help: consider importing this struct
    |
3   + use polars_io::prelude::ParquetAsyncReader;
    |

For more information about this error, try `rustc --explain E0433`.
error: could not compile `polars-plan` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
kalimalrazif commented 3 weeks ago

Hi, I manage to solve de issue but I had to modify my local file /polars-plan-0.41.3/src/plans/functions/count.rs:125:17

Just added the compiler recommendation to the file: 

error[E0433]: failed to resolve: use of undeclared type `ParquetAsyncReader`
   --> /home/nomar/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-plan-0.42.0/src/plans/functions/count.rs:128:17
    |
128 |                 ParquetAsyncReader::from_uri(&path.to_string_lossy(), cloud_options, None).await?;
    |                 ^^^^^^^^^^^^^^^^^^ use of undeclared type `ParquetAsyncReader`
    |
help: a struct with a similar name exists
    |
128 |                 ParquetReader::from_uri(&path.to_string_lossy(), cloud_options, None).await?;
    |                 ~~~~~~~~~~~~~
help: consider importing this struct
    |
3   + use polars_io::prelude::ParquetAsyncReader;
    |

I added use polars_io::prelude::ParquetAsyncReader; to file