search

astronomer / astronomer-providers

Airflow Providers containing Deferrable Operators & Sensors from Astronomer
https://astronomer-providers.rtfd.io/
Apache License 2.0
137 stars 25 forks source link

[GitHub Actions] make base image in integration tests configurable through workflow_dispatch input #996

Closed Lee-W closed 1 year ago

Lee-W commented 1 year ago

Is your feature request related to a problem? Please describe. The base image is not configurable now.

Describe the solution you'd like

Add a "base image" column in the following UI image to configure the base image here

Describe alternatives you've considered

Additional context https://github.com/astronomer/astronomer-providers/pull/991#pullrequestreview-1401116655 Add any other context or screenshots about the feature request here.

Lee-W commented 1 year ago

@pankajkoti But I just found out that we might not be able to downgrade the image on astro. (e.g., we're now on 8.0.0, and we cannot use 7.4.2) In job 8553905019, we received error message Invalid deployment runtime version cannot downgrade to: 7.4.2 from: 8.0.0-alpha1 Do you think we still need it? Or is this behavior only happens in astro breaking change?

pankajkoti commented 1 year ago

yes, it would help us validate our master DAG when new runtime image RCs are out (to test newer versions of Airflow). If we have this, we could just trigger the deployment from the workflow_dispatch with the main branch itself instead of creating a new branch with the required base image and then triggering from that branch.

And your point regarding downgrading is correct, Astro does not support downgrading runtime images. This will help us only when we want to bump it to test new alpha/beta/RC versions

Lee-W commented 1 year ago

And your point regarding downgrading is correct, Astro does not support downgrading runtime images. This will help us only when we want to bump it to test new alpha/beta/RC versions

Does that mean it's ok for us to jump between 8.0.1-alpha1 and 8.0.0? Otherwise, if we can not change the base image back, wouldn't that be a problem?

pankajkoti commented 1 year ago

I am not sure if Astro allows minor versions downgrade. The problem you're mentioning we have today as well when we trigger from a branch with newer version committed in it. Once we bump, we cannot downgrade.

Today, we are not able to deploy back 7.4.2 as we already deployed 8.0.0alpha1.

Let's discuss and gather opinions on this in our standup today 🙂

pankajkoti commented 1 year ago

I think we don't need this. Every upgrade should be tracked in git and we should deploy it from a git branch. So closing this.