dbt-labs / dbt-spark

dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks
https://getdbt.com
Apache License 2.0
395 stars 221 forks source link

[CI/CD Improvements] Migrate to `hatch` and standardize workflows #1027

Open mikealfare opened 5 months ago

mikealfare commented 5 months ago

Short description

As a maintainer, I want to standardize and modernize development tooling, so that I can reduce maintenance issues and focus more on content than form.

Objectives

## Acceptance Criteria
- [ ] Package metadata is defined in `pyproject.toml`
- [ ] `pyproject.toml` contains build configuration that supports `hatch`
- [ ] Relevant scripts or their equivalent are moved into the default `hatch` environment in `pyproject.toml` (e.g. from a `make` file) or are simply retired
- [ ] Test environments use `hatch` instead of `tox`
- [ ] `pytest` configuration is moved into `pyproject.toml`
- [ ] Requirements are updated, mostly removing old dependencies like `tox` and `bumpversion`
- [ ] `precommit` runs the same formatter and linter as runs as `dbt-postgres`
- [ ] All workflows adhere to the style standards below
- [ ] This package has the same CI/CD workflows, run options, and concurrency settings as `dbt-postgres`
- [ ] Branch protection rules are updated, and are limited to a single job, regardless of OS, python version, etc.
- [ ] Integration tests are only configured in one place (e.g. environment variables, etc.)
- [ ] This package is automatically published to dbt's private PyPI daily from `main`; the current workflow is cancelled in favor of the new workflow when this job is resubmitted
- [ ] This package can be published on demand to the public PyPI, GitHub, and Docker for the specified branch; the current workflow is cancelled in favor of the new workflow when this job is resubmitted for the same branch
- [ ] This package is tested against appropriate published packages prior to releasing
- [ ] I can publish to both test and production versions of publishing targets
- [ ] Publishing to PyPI is authorized via Trusted Publishers
- [ ] Legacy workflows are removed

Suggestions

Style standards:

Testing

Security

This should not increase security risk, and potentially could reduce it. Any security risks should be addressed as part of https://github.com/dbt-labs/actions/issues/173.