Closed eladkal closed 1 year ago
pip install apache-airflow-providers-apache-impala[kerberos]
✅hook_params
and deprecated runtime_parameters
.There's however a bug fixed in https://github.com/apache/airflow/pull/32400 which is not exposed though as there's no release of OpenLineage provider yet (method with a bug is never called without the OL provider). Would be nice to include this in this release cycle, I guess?
Thanks for verifying This is not a blocker for this release since the OL integration doesn't work anyway (as OL provider is not yet available) the bug is locallized to OL part. For the moment I see no reason to have RC3, The fix will be included in future release of the provider.
We tested our example, DAGs with the following rc without encountering any error
For #31920 I have a question which I have asked here https://github.com/apache/airflow/pull/31920#issuecomment-1623629476
tested #32089, works as expected
good for me, even if I'd prefer an RC3 to sneak some not-yet-breaking changes in there.
Tested #32126 #31783 #32082 #32120 #32322 works as expected.
Checked #32334 :+1:
vs (6.2.1)
Checked #31787, thanks for your work @eladkal
apache-airflow-providers-alibaba==2.5.0rc2
Tested #32126 #31783 #32082 #32120 #32322 works as expected.
Can you also check 32270 ?
apache-airflow-providers-trino==5.2.0rc2
and tested it against our development trino instance, I can verify the Trino hook reads the jwt file from disk and successfully authenticates with trino.Tested #32126 #31783 #32082 #32120 #32322 works as expected.
Can you also check 32270 ?
Tested with the above versions of providers of Amazon and Microsoft #32270 works as expected. Thanks, @eladkal
Tested #32216 in local Airflow against real GCP Dataproc jobs.
Amazon and Elasticsearch are excluded from RC2 (RC3 will be cut seperatly) The rest of providers are not affected
Tested #31596 https://github.com/apache/airflow/pull/31596, working as expected.
On Sat, Jul 8, 2023 at 12:21 AM eladkal @.***> wrote:
Amazon and Elasticsearch are excluded from RC2 (RC3 will be cut seperatly) The rest of providers are not affected
— Reply to this email directly, view it on GitHub https://github.com/apache/airflow/issues/32389#issuecomment-1625886602, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADDLAHKTQ3NR6BIU7ECEWUDXPBLCXANCNFSM6AAAAAAZ74VIAU . You are receiving this because you were mentioned.Message ID: @.***>
I have tested my changes and all of them work as expected. For #31803, it addresses a part of the problem, but there are still remaining issues in some use cases that need to be resolved, however no need for a new RC, we can release them in next releases, and the merged PR is necessary.
Thank you everyone. Providers are released except Amazon and Elasticsearch which we will have RC3.
I invite everyone to help improve providers for the next release, a list of open issues can be found here.
Hey , I did not test on time the K8S providers -> there is a bug on the release
Hey , I did not test on time the K8S providers -> there is a bug on the release
32458
Provider already released. When fix is merged it will be released in the next provider wave
Provider already released. When fix is merged it will be released in the next provider wave
We could consider yanking it.
We could consider yanking it.
We shouldn't I don't think its a major bug and if we yank kubernetes we must also yank google and cancel amazon rc4 (currently in vote) as they use new feature released in kubernetes provider.
I will cut new release for kubernetes as soon as possible (can't right now because we have another provider vote in progress)
Sure. No problem with that :)
Body
Issue title: Status of testing Providers that were prepared on July 06, 2023
I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers?
Let us know in the comment, whether the issue is addressed.
Those are providers that require testing as there were some substantial changes introduced:
Provider alibaba: 2.5.0rc2
Provider amazon: 8.3.0rc2
ChimeWebhookHook
(#31939): @cjames23ChimeNotifier
(#32222): @cjames23RdsCreateDbInstanceOperator
andRdsDeleteDbInstanceOperator
(#32171): @syedahsnAthenaOperator
(#32186): @vandonr-amzBatchCreateComputeEnvironmentOperator
(#32036): @vandonr-amzGlueJobOperator
to stop the job run when the TI is killed (#32155): @hussein-awalaSageMakerTuningOperator
andSageMakerEndpointOperator
(#32112): @vandonr-amzjob_poll_interval
argument to AWS GlueJobOperator (#32147): @raphaelauvAzureBlobStorageToS3Operator
transfer operator (#32270): @AdaverseUnboundLocalError
while parsing invalids3_url
(#32120): @AdaverseLambdaInvokeFunctionOperator
payload parameter type (#32259): @eladi99ValueError
whenreplace=False
with files already in S3 (#32322): @Adaversedelimiter
param and source object's wildcards in GCS, introducematch_glob
param. (#31261): @shahar1AthenaHook
(#31942): @vandonr-amzon_finish_action
toKubernetesPodOperator
(#30718): @hussein-awalaProvider apache.hive: 6.1.2rc2
Provider apache.impala: 1.1.2rc2
Provider apache.kafka: 1.1.2rc2
Provider apache.livy: 3.5.2rc2
LivyHook
TypeError exception onsession_id
log format (#32051): @joaopamaralProvider cncf.kubernetes: 7.2.0rc2
on_finish_action
toKubernetesPodOperator
(#30718): @hussein-awalaKubernetesPodTrigger
waiting strategy (#31348): @hussein-awalaProvider common.sql: 1.6.0rc2
Provider databricks: 4.3.1rc2
template_fields
ofDatabricksSqlOperator
to support parent class fields (#32253): @eladkalProvider dbt.cloud: 3.2.2rc2
Provider elasticsearch: 5.0.0rc2
Provider google: 10.3.0rc2
on_finish_action
toKubernetesPodOperator
(#30718): @hussein-awalasrc_fmt_configs
to the list of template fields. (#32097): @nathadfieldBigQueryInsertJobOperator
error handling in deferrable mode (#32034): @phanikumvBIGQUERY_JOB_DETAILS_LINK_FMT
inBigQueryConsoleLink
(#31953): @bkossakowskadelimiter
param and source object's wildcards in GCS, introducematch_glob
param. (#31261): @shahar1BigQueryValueCheckOperator
(#31872): @phanikumvProvider hashicorp: 3.4.2rc2
Provider microsoft.azure: 6.2.0rc2
AzureBlobStorageToS3Operator
transfer operator (#32270): @AdaverseProvider postgres: 5.5.2rc2
runtime_parameters
in favor of options inhook_params
(#32345): @JDarDagranProvider snowflake: 4.3.0rc2
Provider trino: 5.2.0rc2
Provider vertica: 3.5.0rc2
The guidelines on how to test providers can be found in
Verify providers by contributors
All users involved in the PRs: @darkag @raphaelauv @blcksrx @vandonr-amz @jhbigler @potiuk @Lee-W @bkossakowska @nathadfield @dcoliversun @phanikumv @pankajastro @aagateuip @joaopamaral @Adaverse @JDarDagran @moiseenkov @Joffreybvn @eladi99 @hussein-awala @notatallshaw-gts @Owen-CH-Leung @shahar1 @eladkal @utkarsharma2 @sunank200 @syedahsn @kristopherkane @cjames23 @ahidalgob @HaeSe0ng
Committer