search

TobikoData / sqlmesh

Efficient data transformation and modeling framework that is backwards compatible with dbt.
https://sqlmesh.com
Apache License 2.0
1.48k stars 123 forks source link

Support Athena #1315

Open nicor88 opened 10 months ago

nicor88 commented 10 months ago

Support for AWS Athena as adapter.

Happy to propose a PR - given some guidelines.

tobymao commented 10 months ago

We don't have plans for this right now, but we're happy to accept a PR.

Come join tobikodata.com/slack to chat with us about it.

Going to close this in favor of https://github.com/TobikoData/sqlmesh/issues/124, but again, happy to help guide you through a pr.

nicor88 commented 10 months ago

@tobymao Athena is based on trino/presto - but there are some internal differences.

See for example https://github.com/dbt-athena/dbt-athena and https://github.com/starburstdata/dbt-trino Most of the SQL commands should be compatible (with some exception e.g. the default catalog), but how the connection is created and how the result is returned back from athena is quite different from trino - I still suggest keeping this as another "issue" and avoid to use the one for trino/presto.

tobymao commented 10 months ago

ok sounds good

alvaro-ponce commented 5 months ago

hi! I am here just wondering how is this feature going on, I am currently giving SQLMesh a try and the Athena adapter is something I would definitively love

Thank you in advance!

pixie79 commented 1 week ago

As someone that builds Greenfield data estates we often start with Athena then switch to other tooling later to manage our data estate when there is volume. It would be great to see Athena supported here like it is with DBT core