search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
35.37k stars 13.81k forks source link

Status of testing Providers that were prepared on June 07, 2022 #24289

Closed potiuk closed 2 years ago

potiuk commented 2 years ago

Body

I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers?

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider alibaba: 2.0.0rc2

The guidelines on how to test providers can be found in

Verify providers by contributors

Committer

vincbeck commented 2 years ago

All my changes are related to doc and/or sample dag. No testing needed

potiuk commented 2 years ago

Cool. Bulk-updated them as "tested" :D

raphaelauv commented 2 years ago

https://github.com/apache/airflow/pull/23981 work

potiuk commented 2 years ago

I also tested and updated all "my" changes

raphaelauv commented 2 years ago

https://github.com/apache/airflow/pull/20578 I have no regression ( not using a new connection or using args of the KPO ) if still using core airlfow kubernetes settings :+1:

potiuk commented 2 years ago

One comment. I am releasing rc2 in a moment (@josh-fell found that our RELEASE notes (part of packages) contained Airflow 2.1+ so I am re-relesing all providers with that one change included (but feel free to continue the testing - the only difference will be those descriptions.

ishiis commented 2 years ago

23724 #24000 is tested.

Dark-Knight11 commented 2 years ago

23849 contains only refactoring of code. No testing needed

pauldalewilliams commented 2 years ago

I've tested #24090 and can confirm it's working as expected.

pankajastro commented 2 years ago

Tested https://github.com/apache/airflow/pull/23839 working as expected.

dstandish commented 2 years ago

@potiuk do you know why k8s is a major?

jonathansp commented 2 years ago

I've tested #23622 and it works as expected.

gilangardya commented 2 years ago

I've tested #24165 (google, BigQueryInsertJobOperator) with composer-2.0.15-airflow-2.2.5, it works as expected. These are the tests I did:

Job type Statement Details 7.0.0 2022.5.18+composer 8.0.0rc1
COPY N/A Copy BigQuery Table :x: :x: :white_check_mark:
EXTRACT N/A To CSV :x: :x: :white_check_mark:
LOAD N/A From CSV :x: :x: :white_check_mark:
QUERY N/A SELECT query :white_check_mark: :white_check_mark: :white_check_mark:
QUERY N/A WRITE_TRUNCATE with partitioned table :white_check_mark: :white_check_mark: :white_check_mark:
QUERY DDL CREATE TABLE AS SELECT :white_check_mark: :white_check_mark: :white_check_mark:
QUERY DML INSERT :white_check_mark: :white_check_mark: :white_check_mark:
QUERY DML MERGE :white_check_mark: :white_check_mark: :white_check_mark:
QUERY DCL GRANT :x: :x: :white_check_mark:
QUERY Procedural DECLARE :x: :x: :white_check_mark:
QUERY Debugging ASSERT :x: :x: :white_check_mark:
QUERY Other EXPORT DATA :x: :x: :white_check_mark:
QUERY Multi-statement DECLARE + INSERT + MERGE :x: :x: :white_check_mark:
nhojjohn commented 2 years ago

I checked https://github.com/apache/airflow/pull/23916 and noticed an problem with it. Feel free to revert it or consider the following PR which fixes the issue https://github.com/apache/airflow/pull/24301

SasanAhmadi commented 2 years ago

I've tested #22802 and it is working as expected.

potiuk commented 2 years ago

I checked #23916 and noticed an problem with it. Feel free to revert it or consider the following PR which fixes the issue #24301

Cool. I will look to it - I am going release rc2 so I will merge it before.

potiuk commented 2 years ago

@potiuk do you know why k8s is a major?

As discussed on slack - mistake :). Will release minor as rc2.

gmcrocetti commented 2 years ago
  1. https://github.com/apache/airflow/pull/23857 works;
  2. https://github.com/apache/airflow/pull/24215 is broken but there's a patch fixing it: https://github.com/apache/airflow/pull/24308;
  3. https://github.com/apache/airflow/pull/23791 I'm currently unable to test it. @eladkal, as the author of the issue would you be willing to test it ?
alexkruc commented 2 years ago

Verified the installation and that everything exists properly for my fixes:

potiuk commented 2 years ago

Thanks @gmcrocetti - as I am releasing rc2 soon, I merged your fix :)

kazanzhy commented 2 years ago

https://github.com/apache/airflow/pull/23767 working as expected

eskarimov commented 2 years ago

Tested #19736 by executing DatabricksRunNowDeferrableOperator and DatabricksSubmitRunDeferrableOperator, both work as expected.

Taragolis commented 2 years ago

Tested #23976 working fine

o-nikolas commented 2 years ago

My changes are testing related and not part of provider code shipped to customers. Though I verified those tests are still passing :+1:

potiuk commented 2 years ago

I re-released the RC2 after few fixes (thanks to those who noticed them) and also corrected a number of documentation issues (that turned to be a bit more complex due to AIP-47 induced changes) - thanks @josh-fell and @deeptansh for raising the issues.

Announcement is about to be sent to the devlist. There is no real need to re-test the already tested changes (I left them checked) as the changes were minimal when it comes to code.

But the not-checked issues also need some ❤️ :).

Summary of changes in RC2:

Have fun with testing (we release on Monday if all is good).

josh-fell commented 2 years ago

All of mine are 👍

nhojjohn commented 2 years ago

tested papermill operator with https://github.com/apache/airflow/pull/24301 in rc2 and works now. Thanks!

potiuk commented 2 years ago

Looking forward for more testing till Monday :).

koconder commented 2 years ago

Tests looking good

potiuk commented 2 years ago

Thanks everyone!. Release is accepted. Proceeding with release.