Status of testing Providers that were prepared on July 09, 2023

[eladkal]

Body

I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers?

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

I marked PRs that were tested in RC2 https://github.com/apache/airflow/issues/32389 as tested.

Provider amazon: 8.3.0rc4

• [ ] Add ChimeWebhookHook (#31939): @cjames23
• [ ] Add ChimeNotifier (#32222): @cjames23
• [ ] Add deferrable mode to S3KeysUnchangedSensor (#31940): @sunank200
• [ ] Add deferrable mode to RdsCreateDbInstanceOperator and RdsDeleteDbInstanceOperator (#32171): @syedahsn
• [X] Add deferrable mode for AthenaOperator (#32186): @vandonr-amz
• [X] Add a deferrable mode to BatchCreateComputeEnvironmentOperator (#32036): @vandonr-amz
• [x] Add deferrable mode in EMR operator and sensor (#32029): @pankajastro
• [X] add async wait method to the "with logging" aws utils (#32055): @vandonr-amz
• [ ] Add custom waiters to EMR Serverless (#30463): @syedahsn
• [X] Add an option to GlueJobOperator to stop the job run when the TI is killed (#32155): @hussein-awala
• [X] deferrable mode for SageMakerTuningOperator and SageMakerEndpointOperator (#32112): @vandonr-amz
• [ ] EKS Create/Delete Nodegroup Deferrable mode (#32165): @syedahsn
• [X] Deferrable mode for ECS operators (#31881): @vandonr-amz
• [ ] Add job_poll_interval argument to AWS GlueJobOperator (#32147): @raphaelauv
• [x] Added AzureBlobStorageToS3Operator transfer operator (#32270): @Adaverse
• [ ] Introduce a base class for aws triggers (#32274): @vandonr-amz
• [X] bugfix: break down run+wait method in ECS operator (#32104): @vandonr-amz
• [X] Handle UnboundLocalError while parsing invalid s3_url (#32120): @Adaverse
• [ ] Fix LambdaInvokeFunctionOperator payload parameter type (#32259): @eladi99
• [X] Bug fix GCSToS3Operator: avoid ValueError when replace=False with files already in S3 (#32322): @Adaverse
• [ ] Deprecate delimiter param and source object's wildcards in GCS, introduce match_glob param. (#31261): @shahar1
• [X] aws waiter util: log status info with error level on waiter error (#32247): @vandonr-amz
• [X] rewrite method used in ecs to fetch less logs (#31786): @vandonr-amz
• [ ] Refactor Eks Create Cluster Operator code (#31960): @syedahsn
• [X] Use a waiter in AthenaHook (#31942): @vandonr-amz
• [X] Add on_finish_action to KubernetesPodOperator (#30718): @hussein-awala
• [X] Add default_deferrable config (#31712): @Lee-W
• [X] deprecate arbitrary parameter passing to RDS hook (#32352): @vandonr-amz
• [ ] quick fix on RDS operator to prevent parameter collision (#32436): @vandonr-amz
• [ ] Remove ability to specify arbitrary hook params in AWS RDS trigger (#32386): @vandonr-amz
• [ ] Only update crawler tags if present in config dict (#32331): @luos-fc

The guidelines on how to test providers can be found in

Verify providers by contributors

All users involved in the PRs: @Lee-W @hussein-awala @sunank200 @eladi99 @raphaelauv @cjames23 @pankajastro @luos-fc @syedahsn @Adaverse @shahar1 @vandonr-amz

Committer

• [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[Adaverse]

Tested #32270, Works as expected!

[Lee-W]

Tested with our example amazon DAG and worked fine.

[pankajastro]

https://github.com/apache/airflow/pull/32029 looks good. thank you!

[eladkal]

Thank you everyone. Amazon provider is released I invite everyone to help improve providers for the next release, a list of open issues can be found here.