search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.79k stars 14.23k forks source link

Status of testing Providers that were prepared on April 09, 2023 #30553

Closed eladkal closed 1 year ago

eladkal commented 1 year ago

I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers?

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider amazon: 7.4.1rc1

The guidelines on how to test providers can be found in

Verify providers by contributors

All users involved in the PRs: @happilyeverafter95 @dolfinus @VladaZakharova @phanikumv @potiuk @tnk-ysk @hussein-awala @vandonr-amz @vanducng @vchiapaikeo @lwyszomi

Committer

vanducng commented 1 year ago

Confirm #30524 bug already got fixed in smtp: 1.0.1rc1. Thanks.

vandonr-amz commented 1 year ago

šŸ‘

vanducng commented 1 year ago

google: 9.0.0rc1 also works well with our google ads and dv360 task.

vchiapaikeo commented 1 year ago

https://github.com/apache/airflow/pull/30543 also still looks fine. Thank you, Elad!

image
potiuk commented 1 year ago

google: 9.0.0rc1 also works well with our google ads and dv360 task.

šŸŽ‰ šŸŽ‰ šŸŽ‰ šŸŽ‰ šŸŽ‰ šŸŽ‰ šŸŽ‰ šŸŽ‰ šŸŽ‰ šŸŽ‰ šŸŽ‰

potiuk commented 1 year ago

Tested mine. All works.

raphaelauv commented 1 year ago 
Use default connection id for KubernetesPodOperator (#28848)

work

hussein-awala commented 1 year ago
tnk-ysk commented 1 year ago

I tested https://github.com/apache/airflow/pull/30441, it works as expected

eladkal commented 1 year ago

https://github.com/apache/airflow/pull/30514 -> it works with the connections created via the Airflow CLI (as I did when I tested the PR before), but not directly with the connections created from the UI, I created https://github.com/apache/airflow/pull/30556 to describe and fix the problem, and https://github.com/apache/airflow/issues/30557 to clean the Azure provider code. Since we can create the connection from the CLI or the environment variables, and even we can edit the UI connection manually once it is created, no need to block the release and create a new one, we can add the fix (and maybe the full clean) to the next release.

since this is the only fix in the release for azure I tend to drop azure from this wave. I think it's preferred to cut RC2 after the fixes are ready

potiuk commented 1 year ago

since this is the only fix in the release for azure I tend to drop azure from this wave. I think it's preferred to cut RC2 after the fixes are ready

We could also apply what we agreed about - shorter voting period for RC2+

We could potentially merge the fix now, prepare RC2 for azure only and release them all together when voting period ends. Might be good time to try this method to see if there is anything to fix in our process to follow this route.

eladkal commented 1 year ago

We could also apply what we agreed about - shorter voting period for RC2+

This is what I plan to do but this will work only if there aren't other changes for azure that were merged in the mean time. I didn't check yet.

potiuk commented 1 year ago

We will need an RC2 for google provider too - (found by @cgadam - fix in #30564).

utkarsharma2 commented 1 year ago

Works well with AstroSDk.

phanikumv commented 1 year ago

30370 works well.

2023-04-10, 14:00:26 UTC] {standard_task_runner.py:55} INFO - Started process 3123 to run task
[2023-04-10, 14:00:26 UTC] {standard_task_runner.py:82} INFO - Running: ['***', 'tasks', 'run', 'example_async_redshift_cluster_management', 'resume_redshift_cluster', 'manual__2023-04-10T13:55:51.894401+00:00', '--job-id', '10', '--raw', '--subdir', 'DAGS_FOLDER/example_redshift_cluster_management.py', '--cfg-path', '/tmp/tmpsyc34091']
[2023-04-10, 14:00:26 UTC] {standard_task_runner.py:83} INFO - Job 10: Subtask resume_redshift_cluster
[2023-04-10, 14:00:26 UTC] {task_command.py:389} INFO - Running <TaskInstance: example_async_redshift_cluster_management.resume_redshift_cluster manual__2023-04-10T13:55:51.894401+00:00 [running]> on host 101451e0c77b
[2023-04-10, 14:00:26 UTC] {redshift_cluster.py:472} INFO - resume_redshift_cluster completed successfully.
[2023-04-10, 14:00:26 UTC] {redshift_cluster.py:473} INFO - Resumed cluster successfully
eladkal commented 1 year ago

microsoft.azure and google will be excluded from rc1. I will cut rc2 as soon as other providers are released.

eladkal commented 1 year ago

Thank you everyone. The google and microsoft.azure providers are excluded from the release and RC2 will be cut for them shortly. other providers are released. I invite everyone to help improve providers for the next release, a list of open issues can be found here.