search

dbt-labs / dbt-athena

The athena adapter plugin for dbt (https://getdbt.com)
https://dbt-athena.github.io
Apache License 2.0
228 stars 100 forks source link

Delta Lake materialization #400

Open redpin-pankaj opened 1 year ago

redpin-pankaj commented 1 year ago

Among the data lake frameworks, this adapter seems to only support iceberg. It would be nice to have delta lake (or maybe hudi) materialization support.

nicor88 commented 1 year ago

Currently, you cannot write delta lake format with Athena, therefore this is currently blocked - I'm aware that trino supports writing as delta, but athena doesn't have that yet, therefore we cannot move on with this implementation. Of course, you can read delta lake tables with athena...

Likinking Athena limitations from the docs: https://docs.aws.amazon.com/athena/latest/ug/delta-lake-tables.html#delta-lake-tables-considerations-and-limitations