apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.39k stars 181 forks source link

Example program gives `mismatched types` error on `ParquetReadOptions` #1021

Open benedetto73 opened 1 week ago

benedetto73 commented 1 week ago

Describe the bug Running the example in here with latest ballista package (39.0.0) gives an error about ParquetReadOptions. Apparently there is some crate that brings in from 35.0.0 and that confuses the compiler. Using ballista 35.0.0 works.

Error log excerpt.

error[E0308]: mismatched types
   --> src/main.rs:18:33
    |
18  |         .read_parquet(filename, ParquetReadOptions::default())
    |          ------------           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `ParquetReadOptions<'_>`, found a different `ParquetReadOptions<'_>`
    |          |
    |          arguments to this method are incorrect
    |
    = note: `ParquetReadOptions<'_>` and `ParquetReadOptions<'_>` have similar names, but are actually distinct types
note: `ParquetReadOptions<'_>` is defined in crate `datafusion`
   --> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/datafusion-39.0.0/src/datasource/file_format/options.rs:185:1
    |
185 | pub struct ParquetReadOptions<'a> {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: `ParquetReadOptions<'_>` is defined in crate `datafusion`
   --> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/datafusion-35.0.0/src/datasource/file_format/options.rs:183:1
    |
183 | pub struct ParquetReadOptions<'a> {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: perhaps two different versions of crate `datafusion` are being used?
note: method defined here
   --> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ballista-0.12.0/src/context.rs:233:18
    |
233 |     pub async fn read_parquet<P: DataFilePaths>(
    |                  ^^^^^^^^^^^^

To Reproduce Run the example program from here with this Cargo.toml (or change datafusion to 35.0.0 to see it work)

[dependencies]
ballista = "0.12.0"
datafusion = "39.0.0"
tokio = "1.38.0"

Expected behavior The program compiles and runs without errors and produces the output on this page.

Additional context None.

milenkovicm commented 1 week ago

@benedetto73 you'd have to use datafusion 35, same version ballista uses, or you could go and update ballista to use datafusion 39