search

duckdb / dbt-duckdb

dbt (http://getdbt.com) adapter for DuckDB (http://duckdb.org)
Apache License 2.0
851 stars 77 forks source link

dbt-duckdb should know about Excel #137

Closed jwills closed 1 year ago

jwills commented 1 year ago

Essentially, any external file format or storage system that can read/write a pandas.DataFrame (or an arrow record reader, or a polars data frame, or a JSON/CSV file) and that has Python library support should be usable as either a) a source or b) an external materialization target in dbt-duckdb.

Reference: dbt-excel.com

davidgasquez commented 1 year ago

Not sure if this exist yet but... is there a world where dbt sources support community made plugins? E.g: Write a subclass of Source with the proper methods (casting to DataFrame/Arrow/...) and you got your source implemented.

jwills commented 1 year ago

@davidgasquez there isn't, but I think that's what this feature needs to support for it to be broadly useful so I was going to try to wire it up