search

sfu-db / connector-x

Fastest library to load data from DB to DataFrames in Rust and Python
https://sfu-db.github.io/connector-x
MIT License
1.94k stars 154 forks source link

rust connector-x 0.3.2 not working with polars 0.33 #543

Closed lidh15 closed 8 months ago

lidh15 commented 1 year ago

What language are you using?

Rust

What version are you using?

0.3.2

What database are you using?

MySQL

What dataframe are you using?

Arrow2, I mean Polars

Can you describe your bug?

you know if I use a polars version different from connector-x is using, the dataframes will have mismatched types (polars::prelude::DataFrame VS polars_core::frame::DataFrame).

What are the steps to reproduce the behavior?

well, simply use polars = "0.33" and connectorx = { version = "0.3.2", features = ["src_mysql", "dst_arrow2"]} in Cargo.toml and try to get_arrow2() and call polars() to return a dataframe

What is the error?

mismatched types expected polars::prelude::DataFrame, found polars_core::frame::DataFrame

lidh15 commented 1 year ago

and it seems that polars 0.33 is bonded with arrow2 0.18 while polars 0.32 is with arrow2 0.17

davidgovea commented 8 months ago

Maybe a new destination is warranted? Seems like polars is using a custom fork of arrow2, so the destination code would look very similar to dst_arrow2

lidh15 commented 8 months ago

the good news is that I modified my codes related to polars so that it could be locked to an older polars version which works with connect-x 0.3.2