search

duckdb / dbt-duckdb

dbt (http://getdbt.com) adapter for DuckDB (http://duckdb.org)
Apache License 2.0
911 stars 86 forks source link

feat: inherit defaults for file formats from location #106

Closed jpmmcneill closed 1 year ago

jpmmcneill commented 1 year ago

solves: https://github.com/jwills/dbt-duckdb/issues/104

Also, I think there is some better handling around location & format (I think one should be required but not both). Happy to submit some changes in a PR.

Originally I was planning on adding json support to the external materialization @jwills, but based on the code I think that's already supported? Happy to add changes to the readme if this is the case.

jwills commented 1 year ago

@jpmmcneill thanks for this man! Apologies for the lag here, I've been on vacation and wasn't logged into GH on my phone.

I started working on a whole reworking of the external stuff to address #104 and also support the partition_by and other more exotic write options that are available in DuckDB's COPY... operations, but I didn't have time to finish it before vacation started: https://github.com/jwills/dbt-duckdb/compare/master...jwills_revamp_external_configs

Would love if you took a look and let me know what you think, I'm also happy to hop on a call with you when I'm back around next week? Can DM you my info in the dbt Slack.

jpmmcneill commented 1 year ago

Hey @jwills happy to take a look. I'll drop you a DM on slack shortly with my details