search

Tomme / dbt-athena

The athena adapter plugin for dbt (https://getdbt.com)
Apache License 2.0
142 stars 79 forks source link

Introducing tagged releases and uploading them to pypi #36

Closed theister closed 1 year ago

theister commented 2 years ago

Hi @Tomme !

Great work on the adapter, it works like a charm and it's awesome to see you're continuing to add new features!

However, the non-standard way of installing your adapter directly from github makes deterministic and verified installs of the adapter a little cumbersome, and makes us hesitant of bringing it into production, even though our Analytics Engineers would love to use dbt to build and maintain Athena tables.

I'm wondering, do you have plans to introduce tagged releases and publish the package to pypi? This would make using and integrating the adapter into our python environments much easier, since users can easily specify pinned sets of python dependencies and test against them, or revert to older version sets if things no longer work/

It seems the pypi package that @lsaletti published under to the name dbt-athena has been abandoned, and its git repository has been deleted, however the info page on pypi links to your project with a recommendation to use it. Have the two of you been in contact or talked about a potential transfer of the pypi project?

Even if a transfer of the pypi project is not possible or desired, I believe most users would be totally fine with installing your adapter using a new name, if it becomes possible to install versioned releases from pypi. For example dbt-athena-adapter is still free as of now.

Please let me know if I can support in any way.

Gatsby-Lee commented 2 years ago

Any comment about this?

dataders commented 2 years ago

also bumping that I'm willing to support this effort as the in-house adapter nerd at dbt Labs. I know open-source maintenance can feel like a burden sometime given everything else we have in life, so please reach out with how (I) we can help!

For example, at any time any person can upload this package to PyPI in their own project with whatever still-available name they want. I learned how to publish to PyPI with this guide and I still open it every time I need to publish a new version.

It actually seems to be what @vanducng is doing with vdn-tools/dbt-athena2, just renaming the code and publishing it manually to PyPI as dbt-athena2.

cc: @Tomme @theister @Gatsby-Lee @Dandandan

nialloriordan commented 2 years ago

Hey I'm happy to help with this. @Tomme The only actions that need to be done for this are as follows:

dataders commented 2 years ago

@lsaletti, would you be amenable to adding @nialloriordan and @Tomme as owners on the dbt-athena PyPI package? For me this is the utopia goal as it's simpler and in alignment with the other adapters.

Change the package name as previously mentioned to dbt-athena-adapter or contact the current owner of dbt-athena in pypi to transfer ownership to you, or delete the old package in pypi and upload the new one. If it is not possible to get in contact with the owner, you can also open a request and ask pypi to change ownership to you. There is an example here: https://github.com/pypa/pypi-support/issues/425#issuecomment-724078630

dataders commented 2 years ago

@Tomme I'm also happy to help with this! How to Publish an Open-Source Python Package to PyPI is the guide i used to learn how to push releases to PyPI (and I still refer to it every time I'm uploading new versions)

  • Create a release on the github repo

  • Create a pypi account: https://test.pypi.org/account/register/

  • Install build and twine: pip install build twine

  • Build the package: python -m build

  • check the package: twine check dist/*

  • upload the package: twine upload dist/*

Tomme commented 1 year ago

Sorry this took so long but we are now publishing to PyPi #119 🥳 with the package name dbt-athena-adapter

dataders commented 1 year ago

hekkin' yeah! @Tomme does https://github.com/dbt-labs/docs.getdbt.com/pull/1856 reflect the change accurately to you?