Automate releases

[drewbanin]

Describe the feature

dbt releases should be as automated as possible. Some ideas:

• Provide a version number (and other info) in structured files (eg. release.md)
• Generate a commit which bumps the version number
• Call the GitHub API to publish a release

Describe alternatives you've considered

Do it manually

[beckjake]

Some notes I have locally regarding future steps:

Two scripts: A tagger and a CI script

Tagger:

• give the script three arguments: a commit to base off, a new version tag, and a release.md file of some sort
• bumpversion to the tag version
• commit + push + tag
• use the github API to create a new release with the contents of release.md, on the tag

CI script:

• run on tag builds. Version == the tag name.
• generate frozen requirements file, save it for use in later steps. The requirements file should not include dbt itself (or dbt-core/plugins).
  • commit this?
  • should tests/merges generate this every time as an artifact for builds to pull
• generate .whl/.tar.gz files for pypi
• (before uploading to pypi!): recursively obtain all the dependency files for the requirements file, like poetry does.
  • this will detect any dependency errors
  • grab the shasums and urls while we're doing this to build the homebrew formula
• upload to testpypi, then pypi
  • build homebrew + push it, triggering its CI process
  • instead of using the homebrew-pypi-poet script, just extract the URLs for what we uploaded, since we already got them for the dependencies
  • fix homebrew-dbt so it handles default homebrew version changes
  • build the docker images + push them to docker hub

Homebrew automation:

• actually push our homebrew changes
• fix the homebrew script to look back multiple changes for commits.
  • alternative: push the first formula change, wait for homebrew-dbt to push its new bottle commit, then push the second formula change
  • alternative: push both the formula changes in one commit

Dockerfile automation:

• should we move the dockerfile generation and release dockerfiles to a separate repository?
• Should the requirements files move there, too?
• Would still generate + push the docker file, like it does for homebrew. But the repository holding the dockerfiles would build the docker image itself in its CI (like homebrew)

[drewbanin]

closing in favor of https://github.com/fishtown-analytics/dbt/issues/2436