duckdb / duckdb-rs

Ergonomic bindings to duckdb for Rust
MIT License
511 stars 113 forks source link

VTab: filter_pushdown support #401

Open joewalnes opened 1 week ago

joewalnes commented 1 week ago

Request to include filter pushdown support for vtab extensions in Rust bindings.

This would allow things like the postgres extension (C++) to be built in Rust - allowing WHERE clause parameters to be propagated. I have a similar need, and would love to use Rust for this.

I'm happy to contribute this, if project is happy with the idea.

Straw-person API proposal

Enabling

impl TableFunction {
    ...
    pub fn supports_filter_pushdown(&self, supports: bool) -> &Self { ... }
    ...
}

pub trait VTab: Sized {
    ...
    fn supports_filter_pushdown() -> bool { false }
    ...
}

(Later: consider renaming the existing supports_pushdown() to supports_projection_pushdown(). In the C++ API it's named more explicitly to avoid confusion.)

Accessing

impl InitInfo {
    ...
    pub fn get_filters(&self) -> [(idx_t, TableFilter)] { ... }
    ...
}

Data model

This is roughly equivalent to C++ API, but far more concise with Rust's algebraic data-types

#[derive(Debug)]
pub enum TableFilter {
    Constant {
        expression_type: ExpressionType,
        value: Value,
    },
    IsNull,
    IsNotNull,
    Or(Vec<TableFilter>),
    And(Vec<TableFilter>),
    StructExtract {
        child_name: String,
        child_filter: Box<TableFilter>,
    },
    Optional(Box<TableFilter>),
}

impl TableFilter {
    pub fn evaluate(&self, value: &Value) -> bool { ... }
}

#[derive(Debug)]
pub enum ExpressionType {
    Equal,
    NotEqual,
    LessThan,
    GreaterThan,
    LessThanOrEqual,
    GreaterThanOrEqual,
}

Note: There are many many more ExpressionTypes in the C++ API, but those 6 are the most practical. eg. see postgres_scanner usage.