[FEATURE] Make Duckdb a core dependency, remove QPD

fugue-project / fugue

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

https://fugue-tutorials.readthedocs.io/

Apache License 2.0

1.98k stars 94 forks source link

[FEATURE] Make Duckdb a core dependency, remove QPD #456

Open goodwanghan opened 1 year ago

goodwanghan commented 1 year ago

QPD was created when there was no good local SQL solution. Duckdb is now very mature and very lightweight. So it's time to totally remove QPD from the core dependency and use DuckDB instead.

Also DuckExecutionEngine should become NativeExecutionEngine. We need to merge these two.

swertz commented 5 months ago

Is it possible that QPD is not maintained anymore? The link pointing to the repository from https://pypi.org/project/qpd/ is broken...

I am having a dependency conflict in my project that traces back to QPD (which requires antlr4-python3-runtime<4.12 >=4.11.1, while other dependencies I need depend on antlr4-python3-runtime==4.9.*), so it would help me very much if fugue dropped the dependency on QPD...