search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
36.85k stars 14.24k forks source link

Status of testing Providers that were prepared on July 09, 2024 #40661

Closed eladkal closed 3 months ago

eladkal commented 3 months ago

Body

I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers?

The guidelines on how to test providers can be found in

Verify providers by contributors

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider amazon: 8.26.0rc1

All users involved in the PRs: @BenediktRamsauer @marcel-martinelli @kacpermuda @uzhastik @jsjasonseba @antoinetavant @theopus @CYarros10 @sc250072 @MaksYermak @boraberke @guillaumeblaquiere @o-nikolas @Lee-W @shahar1 @potiuk @ellism

Committer

potiuk commented 3 months ago

I checked all my changes they look good. One SMALL thing.. I noticed that weaviate changelog is wrong - the 2.0.0 changelog contains also 1.4.2 changes and 1.4.2 is missing from changelog. This is caused by #40194 - where 2.0.0 description replaced 1.4.2 one rather than added it. This should not change the release status of 2.0.0 because changelog is not part of the package and we can fix it by cherry-picking #40663 that fixes it and reganerating the documentation (cc: @eladkal )

kacpermuda commented 3 months ago

I checked all my changes they look good. Thanks !

sc250072 commented 3 months ago

I checked Teradata airflow provider and new feature(s) are working as expected. Thank you!.

MaksYermak commented 3 months ago

I have checked all my changes they work as expected.

vanducng commented 3 months ago

I checked my change for the snowflake provider, it works as expected.

eladkal commented 3 months ago

Weaviate provider will have rc2

marcel-martinelli commented 3 months ago

I checked all my changes on the Databricks operator; it works fine!

gopidesupavan commented 3 months ago

Verified #40492 and #40504 are working fine. Thank you for the release efforts.

dirrao commented 3 months ago

I have verified #40629 and #40506. They are working as expected.

antoinetavant commented 3 months ago

I have verified the Docker operator, both #39620 and #40489 works as expected on airflow:2.8.3-python3.11 as well as airflow:2.9.2-python3.11

BenediktRamsauer commented 3 months ago

I have tested #40642 and it is working.

guillaumeblaquiere commented 3 months ago

Ok for #40560

uzhastik commented 3 months ago

OK for ydb provider and https://github.com/apache/airflow/pull/40390

potiuk commented 3 months ago

I am not sure if that qualifies for RC2, but currently in main AWS auth manager fails compatibility test for 2.8 - i described it in https://github.com/apache/airflow/issues/40684 - cc: @vincbeck, @eladkal

potiuk commented 3 months ago

So I guess with https://github.com/apache/airflow/pull/40690 it means RC2 for Amazon @vincbeck ?

vincbeck commented 3 months ago

Yes!

eladkal commented 3 months ago

~You wish to exclude fab or amazon from rc1?~

Amazon provider will be excluded from rc1

anteverse commented 3 months ago

40518

Checked and working as expected.

shahar1 commented 3 months ago

40479 is ok

jscheffl commented 3 months ago

I just found a serious problem in databricks provider. PR coming in a moment. If PR 40153 is contained in the release then for sure we need to stop and a -1 from my side.

Sent from Outlook for iOShttps://aka.ms/o0ukef

From: Aurelien Didier @.> Sent: Wednesday, July 10, 2024 11:12:28 PM To: apache/airflow @.> Cc: Scheffler Jens (XC-AS/EAE-ADA-T) @.>; Mention @.> Subject: Re: [apache/airflow] Status of testing Providers that were prepared on July 09, 2024 (Issue #40661)

40518https://github.com/apache/airflow/pull/40518

Checked and working as expected.

— Reply to this email directly, view it on GitHubhttps://github.com/apache/airflow/issues/40661#issuecomment-2221482840, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWVTFDO7LONFW5AE4BDKVGLZLWPTZAVCNFSM6AAAAABKSHNSR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRRGQ4DEOBUGA. You are receiving this because you were mentioned.Message ID: @.***>

jscheffl commented 3 months ago

Dont see my previous email, -1 for databricks, raised a fix PR #40708 but would rather recomment to roll back #40153

Sent from Outlook for iOShttps://aka.ms/o0ukef

From: Elad Kalif @.> Sent: Wednesday, July 10, 2024 6:32:10 PM To: apache/airflow @.> Cc: Scheffler Jens (XC-AS/EAE-ADA-T) @.>; Mention @.> Subject: Re: [apache/airflow] Status of testing Providers that were prepared on July 09, 2024 (Issue #40661)

I will exclude fab from rc1

— Reply to this email directly, view it on GitHubhttps://github.com/apache/airflow/issues/40661#issuecomment-2220976653, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWVTFDLUQO5ARSR2PFJPOR3ZLVOYVAVCNFSM6AAAAABKSHNSR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRQHE3TMNRVGM. You are receiving this because you were mentioned.Message ID: @.***>

jscheffl commented 3 months ago

Ah, emails are getting to ticket and not to devlist :-D

As far as I see the PR #40153 is not in the providers-cut? Then it is OK...

pankajkoti commented 3 months ago

yes, #40153 is not in this providers wave

dabla commented 3 months ago

PR 40615 partially fixes the issue, it doesn't fix it for the JdbcHook as there the driver_path and driver_class property causes the same issue as placeholder did. Also mentioned this in original issue.

potiuk commented 3 months ago

PR 40615 partially fixes the issue, it doesn't fix it for the JdbcHook as there the driver_path and driver_class property causes the same issue as placeholder did. Also mentioned this in original issue.

Yeah. Good catch @dabla -> I think it's not a blocker for this release, it's not a regression comparing to already released common.sql and we can always fix it in the next round (and it has rather small-ish impact).

eladkal commented 3 months ago

@dabla can you raise PR with a fix for it?

dabla commented 3 months ago

@dabla can you raise PR with a fix for it?

Hey Elad, yes I already have a fix for it, but would like to see PR #40665 merged first as this will allow me to have a cleaner solution. Unless you want this fix first then I will have to refactor the fix in #40665 afterwards, that's also possible. It's up to you

potiuk commented 3 months ago

I think it's not a regression and can be easily fixed in the next round if you ask me :) - there are few other things that @dabla is contributing in this area https://github.com/apache/airflow/pull/40665 and likely they could be done in next release ?

potiuk commented 3 months ago

Similar answer :)

dabla commented 3 months ago

Same for me, if you want the fix @eladkal, it can be done now/today...

dabla commented 3 months ago

I already started with the PR for this fix, can merge #40665 afterwards, not such a big issue anyway

eladkal commented 3 months ago

Thank you everyone. Providers are released (excluding amazon and weaviate)

I invite everyone to help improve providers for the next release, a list of open issues can be found here.

dabla commented 3 months ago

@dabla can you raise PR with a fix for it?

https://github.com/apache/airflow/pull/40751