Status of testing Providers that were prepared on September 21, 2024

[eladkal]

Body

I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers?

The guidelines on how to test providers can be found in

Verify providers by contributors

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider airbyte: 4.0.0rc1

• [ ] Provider Airbyte: breaking change, update provider to use Airbyte API Python SDK (#41122): @marcosmarxm
• [x] Fix wrong casing in airbyte hook. (#42170): @potiuk Linked issues:
  • [x] Linked Issue #42154: @jscheffl
• [x] Pin airbyte-api to 0.51.0 (#42154) (#42155): @jscheffl Linked issues:
  • [x] Linked Issue #42154: @jscheffl

    Provider alibaba: 2.9.1rc1

• [ ] log handler deprecated filename_template argument removal (#41552): @dirrao

  Provider amazon: 8.29.0rc1

• [ ] Adding support for volume configurations in ECSRunTaskOperator (#42087): @srehman420
• [ ] Openlineage s3 to redshift operator integration (#41575): @Artuz37
• [ ] ECS Exec: Drop params that aren't compatible with EC2 (#42228): @o-nikolas Linked issues:
  • [ ] Linked Issue #41824: @AdamStrLSEC
• [ ] Fix GlueDataBrewStartJobOperator template fields (#42073): @ferruzzi
• [ ] Validate aws service exceptions in waiters (#41941): @gopidesupavan Linked issues:
  • [ ] Linked Issue #41918: @sarkch
• [ ] Fix treatment of "#" in S3Hook.parse_s3_url() (#41796): @GlenboLake
• [ ] fix: remove part of openlineage extraction from S3ToRedshiftOperator (#41631): @kacpermuda
• [ ] filename template arg in providers file task handlers backward compitability support (#41633): @dirrao Linked issues:
  • [ ] Linked Issue #41552: @dirrao
• [ ] fix: custom query should have precedence over default query in RedshiftToS3Operator (#41634): @kacpermuda Linked issues:
  • [ ] Linked Issue #37861: @okirialbert
• [ ] Actually move saml to amazon provider (mistakenly added in papermill) (#42148): @potiuk Linked issues:
  • [ ] Linked Issue #42137: @potiuk
• [ ] Use base aws classes in AWS Glue DataBrew Operators/Triggers (#41848): @gopidesupavan Linked issues:
  • [ ] Linked Issue #35278: @Taragolis
• [ ] Move register_views to auth manager interface (#41777): @vincbeck
• [ ] Limit watchtower as depenendcy as 3.3.0 breaks main. (#41612): @potiuk
• [ ] log handler deprecated filename_template argument removal (#41552): @dirrao

  Provider apache.hdfs: 4.5.1rc1

• [ ] log handler deprecated filename_template argument removal (#41552): @dirrao

  Provider apache.impala: 1.5.1rc1

• [ ] Make kerberos an optional and devel dependency for impala and fab (#41616): @potiuk

  Provider apache.livy: 3.9.1rc1

• [ ] bugfix/livy-set-base-url (#41658): @harryshi10

  Provider apache.spark: 4.11.0rc1

• [ ] Add kerberos related connection fields(principal, keytab) on SparkSubmitHook (#40757): @seyoon-lim

  Provider cloudant: 4.0.0rc1

• [ ] Switch cloudant provider from cloudant library to ibmcloudant library (#41555): @topherinternational Linked issues:
  • [ ] Linked Issue #21004: @uranusjr

    Provider cncf.kubernetes: 8.4.2rc1

• [ ] Deprecated configuration removed (#42129): @dirrao

  Provider common.io: 1.4.1rc1

• [ ] Protect against None components of universal pathlib xcom backend (#41921): @potiuk Linked issues:
  • [ ] Linked Issue #41723: @potiuk

    Provider common.sql: 1.17.0rc1

• [ ] Generalize caching of connection in DbApiHook to improve performance (#40751): @dabla
• [ ] feat: log client db messages for provider postgres (#40171): @dondaum Linked issues:
  • [ ] Linked Issue #40116: @hpereira98

    Provider databricks: 6.10.0rc1

• [ ] [FEAT] databricks repair run with reason match and appropriate new settings (#41412): @gaurav7261
• [ ] Removed deprecated method reference airflow.www.auth.has_access when min airflow version >= 2.8.0 (#41747): @dirrao

  Provider docker: 3.14.0rc1

• [ ] Added logging device and logging device options (#41416): @geraj1010 Linked issues:
  • [ ] Linked Issue #40533: @xjchew
• [ ] Add retrieve output docker swarm operator (#41531): @rgriffier

  Provider elasticsearch: 5.5.1rc1

• [ ] Fix ElasticSearch SQLClient deprecation warning (#41871): @Owen-CH-Leung
• [ ] Generalize caching of connection in DbApiHook to improve performance (#40751): @dabla
• [ ] filename template arg in providers file task handlers backward compitability support (#41633): @dirrao Linked issues:
  • [ ] Linked Issue #41552: @dirrao
• [ ] log handler deprecated filename_template argument removal (#41552): @dirrao

  Provider fab: 1.4.0rc1

• [ ] Add FAB migration commands (#41804): @ephraimbuddy
• [ ] Separate FAB migration from Core Airflow migration (#41437): @ephraimbuddy
• [ ] Deprecated kerberos auth removed (#41693): @dirrao
• [ ] Deprecated configuration removed (#42129): @dirrao
• [ ] Move is_active user property to FAB auth manager (#42042): @vincbeck
• [ ] Move register_views to auth manager interface (#41777): @vincbeck
• [ ] Revert "Provider fab auth manager deprecated methods removed (#41720)" (#41960): @potiuk
• [ ] provider fab auth manager deprecated methods removed (#41720): @dirrao
• [ ] Make kerberos an optional and devel dependency for impala and fab (#41616): @potiuk

  Provider google: 10.23.0rc1

• [ ] Add ability to create Flink Jobs in dataproc cluster (#42342): @VladaZakharova
• [ ] Add new Google Search 360 Reporting Operators (#42255): @molcay
• [ ] Add return_immediately parameter to PubSubPullSensor class (#41842): @arnaubadia Linked issues:
  • [ ] Linked Issue #41838: @arnaubadia
  • [ ] Linked Issue #23231: @eladkal
• [ ] Add parent_model param in UploadModelOperator (#42091): @jx2lee Linked issues:
  • [ ] Linked Issue #41194: @GLaDAP
• [ ] Add DataflowStartYamlJobOperator (#41576): @moiseenkov
• [ ] Add RunEvaluationOperator for Google Vertex AI Rapid Evaluation API (#41940): @CYarros10
• [ ] Add CountTokensOperator for Google Generative AI CountTokensAPI (#41908): @CYarros10
• [ ] Add Supervised Fine Tuning Train Operator, Hook, Tests, Docs (#41807): @CYarros10
• [ ] Minor fixes to ensure successful Vertex AI LLMops pipeline (#41997): @CYarros10
• [ ] Exclude partition from BigQuery table name (#42130): @moiseenkov
• [ ] [Fix #41763]: Redundant forward slash in SFTPToGCSOperator when destination_path is not specified or have default value (#41928): @Mayuresh16 Linked issues:
  • [ ] Linked Issue #41763: @mpospis
• [ ] Fix poll_interval in GKEJobTrigger (#41712): @gopidesupavan Linked issues:
  • [ ] Linked Issue #41705: @bwatan
• [ ] update pattern for dataflow job id extraction (#41794): @lukas-mi
• [ ] Fix 'do_xcom_push' and 'get_logs' functionality for KubernetesJobOperator (#40814): @MaksYermak
• [ ] Mark VertexAI AutoMLText deprecation (#42251): @olegkachur-e
• [ ] Remove system test for deprecated Google analytics operators (#41946): @moiseenkov
• [ ] Unpin google-cloud-bigquery package version for Google provider (#41839): @MaksYermak
• [x] Move away from deprecated DAG.following_schedule() method (#41773): @jscheffl
• [ ] log handler deprecated filename_template argument removal (#41552): @dirrao

  Provider jdbc: 4.5.1rc1

• [ ] Generalize caching of connection in DbApiHook to improve performance (#40751): @dabla

  Provider microsoft.azure: 10.5.0rc1

• [ ] Allow custom api versions in MSGraphAsyncOperator (#41331): @dabla
• [ ] Add callback to process Azure Service Bus message contents (#41601): @perry2of5
• [ ] log handler deprecated filename_template argument removal (#41552): @dirrao

  Provider microsoft.mssql: 3.9.1rc1

• [ ] Generalize caching of connection in DbApiHook to improve performance (#40751): @dabla

  Provider mongo: 4.2.1rc1

• [x] Adjust typing in Mongo hook to prevent mypy errors (#42354): @jscheffl
• [ ] Improve Mongo connection validation and UI (#41717): @topherinternational Linked issues:
  • [ ] Linked Issue #41371: @topherinternational

    Provider mysql: 5.7.1rc1

• [ ] Generalize caching of connection in DbApiHook to improve performance (#40751): @dabla

  Provider odbc: 4.7.1rc1

• [ ] Generalize caching of connection in DbApiHook to improve performance (#40751): @dabla

  Provider openai: 1.4.0rc1

• [ ] feat(providers/openai): support batch api in hook/operator/trigger (#41554): @josix Linked issues:
  • [ ] Linked Issue #41336: @josix

    Provider openlineage: 1.12.0rc1

• [ ] [feat: notify about potential serialization failures when sending DagRun, don't serialize unnecessary params, guard listener for exceptions (#41690)](https://github.com/apache/airflow/pull/4169
• [ ] fix: revert behavior of flattening lists in OpenLineage's InfoJsonEncodable (#41786): @mobuchowski
• [ ] chore: bump OL provider dependencies versions (#42059): @kacpermuda
• [ ] move to dag_run.logical_date from execution date in OpenLineage provider (#41889): @mobuchowski
• [ ] Unify DAG schedule args and change default to None (#41453): @uranusjr

  Provider papermill: 3.8.1rc1

• [ ] Actually move saml to amazon provider (mistakenly added in papermill) (#42148): @potiuk Linked issues:
  • [ ] Linked Issue #42137: @potiuk
• [ ] Make SAML a required dependency of Amazon provider (#42137): @potiuk

  Provider postgres: 5.13.0rc1

• [ ] feat: log client db messages for provider postgres (#40171): @dondaum Linked issues:
  • [ ] Linked Issue #40116: @hpereira98
• [ ] Generalize caching of connection in DbApiHook to improve performance (#40751): @dabla

  Provider snowflake: 5.7.1rc1

• [ ] Update snowflake naming for account names and locators. (#41775): @JDarDagran

All users involved in the PRs: @gaurav7261 @ephraimbuddy @jscheffl @harryshi10 @dabla @moiseenkov @marcosmarxm @arnaubadia @o-nikolas @Artuz37 @geraj1010 @perry2of5 @josix @kacpermuda @dondaum @Mayuresh16 @olegkachur-e @ferruzzi @mob

Committer

• [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[jscheffl]

Signed-off my fixes I contributed. They were mainly related to failing main, if pytests are fine then all is good.