chdb-io / chdb

chDB is an in-process OLAP SQL Engine 🚀 powered by ClickHouse
https://clickhouse.com/docs/en/chdb
Apache License 2.0
2k stars 72 forks source link

Filter pushdown in PyReader #249

Open delta003 opened 1 month ago

delta003 commented 1 month ago

Use case

I'd like to use chdb as a query engine for a custom storage engine implemented in Python.

Describe the solution you'd like

Pass some kind of filters to PyReader#read method, e.g. https://arc.net/l/quote/mshaakfn

Additional context

I was looking for a solution that I can use with an existing Python implementation and found chdb.PyReader. My guess is that most of pushdown logic exists and it's a matter of wiring it? I'll take a closer look at the code, and can contribute the feature, but wanted to ask first before wasting time.

auxten commented 1 month ago

ClickHouse do have filter pushdown features, but for PyReader#read I didn't implement that. You are very very welcomed if you are interested to implement that!