The most popular ClickHouse plugin for Airflow. 🔝 Top-1% downloads on PyPI: https://pypi.org/project/airflow-clickhouse-plugin! Based on mymarilyn/clickhouse-driver.
MIT License
144
stars
36
forks
source link
Automate versions extraction in PyPI publishing file #81
In the current PyPI publishing workflow file, versions of the plugin, Airflow, and Python are hardcoded in multiple places. This introduces a risk of version mismatches, when releasing new versions. For instance, this is evident in the following section of the file:
To mitigate this risk, I propose implementing one or several of the following actions:
Comprehensive Version Tracking. Identify and list in the comments all locations within the workflow file where versions are hardcoded to ensure none are missed during updates.
Introduction of Workflow Variables. Introduce workflow variables to define the versions once and reference them throughout the file instead of hardcoding them repeatedly.
Automation of Version Extraction. Extract versions automatically to minimize manual intervention:
Extract plugin version from pyproject.toml.
Derive Airflow's latest supported version from pip install.
Use the latest available Python version: python: "3.x", and refer to it via python --version.
These proposed changes pertain specifically to the PyPI publishing workflow and do not affect the unit/integration tests dedicated workflow.
I'm open to hearing your thoughts and suggestions on how best to address this issue. If you have any alternative ideas or references to existing GitHub projects, please feel free to share them in the comments below.
In the current PyPI publishing workflow file, versions of the plugin, Airflow, and Python are hardcoded in multiple places. This introduces a risk of version mismatches, when releasing new versions. For instance, this is evident in the following section of the file:
https://github.com/bryzgaloff/airflow-clickhouse-plugin/blob/1b6179baaab5ae4406a206dd5d2f033ca6347186/.github/workflows/publish-to-pypi.yml#L102-L110
Proposed Solutions
To mitigate this risk, I propose implementing one or several of the following actions:
pyproject.toml.pip install.python: "3.x", and refer to it viapython --version.These proposed changes pertain specifically to the PyPI publishing workflow and do not affect the unit/integration tests dedicated workflow.
I'm open to hearing your thoughts and suggestions on how best to address this issue. If you have any alternative ideas or references to existing GitHub projects, please feel free to share them in the comments below.