apache / datafusion-ballista

Apache DataFusion Ballista Distributed Query Engine
https://datafusion.apache.org/ballista
Apache License 2.0
1.39k stars 181 forks source link

Ballista example project does not build #482

Open garro95 opened 2 years ago

garro95 commented 2 years ago

Describe the bug A clear and concise description of what the bug is. The example at https://github.com/apache/arrow-datafusion/blob/master/ballista/rust/client/README.md#executing-a-query does not build

To Reproduce Steps to reproduce the behavior:

Expected behavior A clear and concise description of what you expected to happen. A successful build

Additional context Add any other context about the problem here.

The error is

error: failed to select a version for `zstd-safe`.
    ... required by package `zstd v0.9.0+zstd.1.5.0`
    ... which satisfies dependency `zstd = "^0.9"` of package `parquet v6.1.0`
    ... which satisfies dependency `parquet = "^6.1.0"` of package `datafusion v6.0.0`
    ... which satisfies dependency `datafusion = "^6.0.0"` of package `ballista v0.6.0`
    ... which satisfies dependency `ballista = "^0.6"` of package `ballista-example v0.1.0`
versions that meet the requirements `=4.1.1` are: 4.1.1+zstd.1.5.0

all possible versions conflict with previously selected packages.

  previously selected package `zstd-safe v4.1.4+zstd.1.5.2`
    ... which satisfies dependency `zstd-safe = "=4.1.4"` of package `zstd v0.10.0+zstd.1.5.2`
    ... which satisfies dependency `zstd = "^0.10"` of package `parquet v9.0.2`
    ... which satisfies dependency `parquet = "^9.0.0"` of package `datafusion v7.0.0`
    ... which satisfies dependency `datafusion = "^7.0"` of package `ballista-example v0.1.0`

failed to select a version for `zstd-safe` which could resolve this conflict
garro95 commented 2 years ago

After selecting datafusion = "6.0" instead of the 7.0 suggested, I encountered a new error:

error[E0599]: no function or associated item named `from_str_unchecked` found for struct `proc_macro2::Literal` in the current scope
   --> /home/garro95/.cargo/registry/src/github.com-1ecc6299db9ec823/quote-1.0.11/src/runtime.rs:301:41
    |
301 |         let literal = unsafe { Literal::from_str_unchecked(repr) };
    |                                         ^^^^^^^^^^^^^^^^^^ function or associated item not found in `proc_macro2::Literal`
comphead commented 2 years ago

Hi @garro95. Please attach a link to your github repo, where you get the issue above.

garro95 commented 2 years ago

https://github.com/garro95/ballista-example

comphead commented 2 years ago

Looks like related to apache/arrow-datafusion#1498

garro95 commented 2 years ago

Well, in part probably. The first issue is that the dependencies

ballista = "0.6"
datafusion = "7.0"

as listed in https://github.com/apache/arrow-datafusion/blob/master/ballista/rust/client/README.md#executing-a-query do not work together, giving the compilation error that you can see in my first post. The issue I encountered after fixing that, actually seems related to apache/arrow-datafusion#1498

comphead commented 2 years ago

I was able to build the project with


[dependencies]
ballista = "0.6"
datafusion = "6.0.0"
futures = "0.3"
num_cpus = "1.13.0"
prost = "0.10"
tokio = { version = "1.0", features = ["macros", "rt", "rt-multi-thread", "sync", "parking_lot"] }
tonic = "0.7"
quote = "=1.0.10"

But in that case main.rs won't be compiled anyway. Perhaps @andygrove knows when its expected new ballista release with datafusion 7.0.0

Alternatively you can try ballista and datafusion by cloning the entire datafusion repo and building it locally.