Status of testing Providers that were prepared on November 14, 2024

[eladkal]

Body

I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers?

The guidelines on how to test providers can be found in

Verify providers by contributors

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider amazon: 9.1.0rc4

• [ ] feat: add OpenLineage support for RedshiftToS3Operator (#41632): @kacpermuda Linked issues:
  • [ ] Linked Issue #41575: @Artuz37
• [ ] Add SageMakerProcessingSensor (#43144): @vVv-AA
• [ ] Make RedshiftDataOperator handle multiple queries (#42900): @jroachgolf84
• [ ] fix(providers/amazon): alias is_authorized_dataset to is_authorized_asset (#43470): @Lee-W
• [ ] Remove returns in final clause of athena hooks (#43426): @yangyulely Linked issues:
  • [ ] Linked Issue #43274: @iritkatriel
• [ ] fix: replace \s with space (#43849): @LyndonFan Linked issues:
  • [ ] Linked Issue #38864: @vincbeck
  • [ ] Linked Issue #39252: @Taragolis
  • [ ] Linked Issue #39428: @romsharon98
  • [ ] Linked Issue #38734: @dabla
• [ ] Fix HttpToS3Operator throws exception if s3_bucket parameter is not passed (#43828): @kunaljubce Linked issues:
  • [ ] Linked Issue #43379: @kostiantyn-lab
• [ ] Fix awslogs_stream_prefix pattern (#43138): @pyrr Linked issues:
  • [ ] Linked Issue #43130: @pyrr
• [ ] Check if awslogs_stream_prefix already ends with container_name (#43724): @pyrr Linked issues:
  • [ ] Linked Issue #43138: @pyrr
• [ ] bugfix description should be optional for openlineage integration with AthenaOperator (#43576): @Udbv
• [ ] Decouple volume_configurations from capacity_provider_strategy (#43047): @pyrr Linked issues:
  • [ ] Linked Issue #43046: @pyrr
• [ ] GlueJobOperator: add option to wait for cleanup before returning job status (#43688): @eladkal
• [ ] Resolve GlueJobTrigger serialization bug causing verbose to always be True (#43622): @jimwbaldwin
• [ ] Remove returns in final clause of S3ToDynamoDBOperator (#43456): @yangyulely Linked issues:
  • [ ] Linked Issue #43274: @iritkatriel
• [ ] Remove sqlalchemy-redshift dependency (#43271): @mobuchowski
• [ ] feat(providers/amazon): Use asset in common provider (#43110): @Lee-W
• [ ] Restrict looker-sdk version 24.18.0 and microsoft-kiota-http 1.3.4 (#42954): @gopidesupavan
• [ ] Limit mypy-boto3-appflow (#43436): @eladkal
• [ ] Standard provider python operator (#42081): @gopidesupavan
• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42
• [ ] remove all deprecations cncf.kubernetes (#43689): @romsharon98
• [ ] Fix docstring for AthenaTrigger (#43616): @eladkal

  Provider apache.beam: 5.9.1rc1

• [ ] Standard provider python operator (#42081): @gopidesupavan

  Provider apache.drill: 2.8.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider apache.druid: 3.12.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider apache.hive: 8.2.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42
• [x] Explain how to use uv with airflow virtualenv and make it works (#43604): @potiuk Linked issues:
  • [x] Linked Issue #43200: @potiuk
• [ ] Move uncompress_file function from airflow.utils to Hive provider (#43526): @kaxil

  Provider apache.impala: 1.5.2rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider apache.pinot: 4.5.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider apache.spark: 4.11.3rc1

• [ ] Standard provider python operator (#42081): @gopidesupavan

  Provider apprise: 1.4.1rc1

• [ ] Standard provider bash operator (#42252): @gopidesupavan
• [ ] Unify DAG schedule args and change default to None (#41453): @uranusjr

  Provider atlassian.jira: 2.7.1rc1

• [ ] Standard provider bash operator (#42252): @gopidesupavan

  Provider celery: 3.8.4rc1

• [ ] AIP-72: Remove DAG pickling (#43667): @kaxil
• [ ] Standard provider python operator (#42081): @gopidesupavan

  Provider cloudant: 4.0.3rc1

• [ ] Bump uv to 0.4.28 (#43451): @kaxil

  Provider cncf.kubernetes: 10.0.0rc1

• [ ] remove all deprecations cncf.kubernetes (#43689): @romsharon98
• [ ] Change default value of namespace in task.kubernetes to be None (#43402): @romsharon98
• [ ] Add random_name_suffix to SparkKubernetesOperator (#43800) (#43847): @mrk-andreev Linked issues:
  • [ ] Linked Issue #43800: @Flametaa
• [ ] terminate kubernetes watch in case of unknown error while flushing queue (#43645): @pavansharma36 Linked issues:
  • [ ] Linked Issue #43440: @iw-pavan
• [ ] Update 'namespace' priority for 'find_pod' function (#43762): @MaksYermak
• [ ] AIP-72: Remove DAG pickling (#43667): @kaxil

  Provider common.compat: 1.2.2rc1

• [ ] serialize asset/dataset timetable conditions in OpenLineage info also for older supported Airflow 2 versions (#43434): @mobuchowski
• [ ] Standard provider python operator (#42081): @gopidesupavan

  Provider common.sql: 1.20.0rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider databricks: 6.13.0rc1

• [ ] Add on_kill equivalent to Databricks SQL Hook to cancel timed out queries (#42668): @R7L208 Linked issues:
  • [ ] Linked Issue #42115: @R7L208
• [ ] Added support for job_parameters and dbt_commands in DatabricksRunNow Operator (#43895): @pranshupand-db
• [ ] Enable workload identity authentication for the Databricks provider (#41639): @basvandriel Linked issues:
  • [ ] Linked Issue #41586: @basvandriel
• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider dbt.cloud: 3.11.2rc1

• [ ] Added condition to check if it is a scheduled save or rerun (#43453): @krzysztof-kubis Linked issues:
  • [ ] Linked Issue #43347: @krzysztof-kubis

    Provider docker: 3.14.1rc1

• [ ] Fix logs with leading spaces in the Docker operator (#33692) (#43840): @mrk-andreev
• [ ] Standard provider python operator (#42081): @gopidesupavan

  Provider elasticsearch: 5.5.3rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider exasol: 4.6.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider fab: 1.5.1rc1

• [ ] fab_auth_manager: allow get_user method to return the user authenticated via Kerberos (#43662): @brouberol

  Provider google: 10.26.0rc1

• [ ] Add support for IAM database authentication for CloudSQL connection (#43631): @MaksYermak
• [ ] Provide option to force_delete for GCSToBigQueryOperator (#43785): @nathadfield
• [ ] Unify reattach_states parameter logic across BigQuery operators (#43259): @moiseenkov Linked issues:
  • [ ] Linked Issue #40664: @VladaZakharova
• [ ] Remove non-existing field from the ListCustomTrainingJobOperator's template_fields (#43924): @moiseenkov
• [ ] Fix validating parent_model parameter in UploadModelOperator (#43473): @VladaZakharova
• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42
• [ ] Remove Airflow 2.1 compat code in Google provider (#43952): @uranusjr
• [x] Explain how to use uv with airflow virtualenv and make it works (#43604): @potiuk Linked issues:
  • [x] Linked Issue #43200: @potiuk
• [ ] Standard provider python operator (#42081): @gopidesupavan
• [ ] Bump google-ads version to use v18 by default (#43474): @VladaZakharova

  Provider http: 4.13.3rc1

• [ ] Fix(http) bug, pass request_kwargs to HttpHooK.run (#43459): @childe
• [x] Limit temporarily aiohttp to < 3.11.0 (#44006): @potiuk

  Provider jdbc: 4.5.3rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider microsoft.azure: 11.1.0rc1

• [ ] Add copy_object functionality for wasbhook (#43037): @kunaljubce Linked issues:
  • [ ] Linked Issue #42497: @kunaljubce
• [ ] Fix Power BI trigger testcase (#43494): @ambika-garg
• [ ] Add min version to ipykernel,scrapbook, pywinrm (#43603): @rawwar

  Provider microsoft.mssql: 3.9.2rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider microsoft.winrm: 3.6.1rc1

• [ ] Move responsibility to run a command from WinRMOperator to WinRMHook (#43646): @dabla

  Provider mysql: 5.7.4rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42
• [x] Explain how to use uv with airflow virtualenv and make it works (#43604): @potiuk Linked issues:
  • [x] Linked Issue #43200: @potiuk

    Provider odbc: 4.8.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider openlineage: 1.14.0rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42
• [ ] add ProcessingEngineRunFacet to OL DAG Start event (#43213): @mobuchowski
• [ ] serialize asset/dataset timetable conditions in OpenLineage info also for older supported Airflow 2 versions (#43434): @mobuchowski
• [ ] openlineage: accept whole config when instantiating OpenLineageClient (#43740): @JDarDagran
• [x] Temporarily limit openlineage to <1.24.0 (#43732): @potiuk
• [ ] Standard provider python operator (#42081): @gopidesupavan

  Provider oracle: 3.12.1rc1

• [ ] Fix oracle bulk insert issue when leftover chunk is empty (#43467): @romsharon98
• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider pagerduty: 3.8.1rc1

• [ ] Standard provider bash operator (#42252): @gopidesupavan
• [ ] Purge existing SLA implementation (#42285): @ferruzzi

  Provider papermill: 3.8.2rc1

• [ ] Add min version to ipykernel,scrapbook, pywinrm (#43603): @rawwar

  Provider pinecone: 2.1.1rc1

• [ ] Add source_tag to PineconeHook (#43960): @TJaniF

  Provider postgres: 5.14.0rc1

• [ ] Add AWS Redshift Serverless support to PostgresHook (#43669): @topherinternational
• [ ] Fix PostgresHook bug when getting AWS Redshift Serverless credentials (#43807): @topherinternational Linked issues:
  • [ ] Linked Issue #43669: @topherinternational
• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider presto: 5.7.0rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider slack: 8.9.2rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider smtp: 1.8.1rc1

• [ ] Standard provider bash operator (#42252): @gopidesupavan
• [ ] Purge existing SLA implementation (#42285): @ferruzzi
• [ ] Unify DAG schedule args and change default to None (#41453): @uranusjr

  Provider snowflake: 5.8.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42
• [ ] Standard provider python operator (#42081): @gopidesupavan

  Provider sqlite: 3.9.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider standard: 0.0.1rc1

• [ ] Airflow Standard Provider (#41564): @romsharon98

  Provider teradata: 2.6.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider trino: 5.9.0rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider vertica: 3.9.1rc1

• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

    Provider ydb: 2.0.0rc1

• [ ] Migrate YDB Operator to new DBAPI (#43784): @vgvoleg
• [ ] Add support for semicolon stripping to DbApiHook, PrestoHook, and TrinoHook (#41916): @Illumaria Linked issues:
  • [ ] Linked Issue #34828: @MikeWallis42

All users involved in the PRs: @basvandriel @TJaniF @vgvoleg @kacpermuda @pyrr @Illumaria @MaksYermak @pranshupand-db @kunaljubce @romsharon98 @nathadfield @kaxil @Udbv @childe @krzysztof-kubis @LyndonFan @mobuchowski @ferruzzi @brou

Committer

• [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[potiuk]

I checked all my changes - most of them were docs and dependencies. Dependencies look good (openlineage change of mine was overwritten later by @JDarDagran - so all good there as well.

So everything looks good from my side. 👍

[JDarDagran]

I checked all my changes - most of them were docs and dependencies. Dependencies look good (openlineage change of mine was overwritten later by @JDarDagran - so all good there as well.

Yup, confirming my changes from from #43740 work as expected.

[jimwbaldwin]

https://github.com/apache/airflow/pull/43622: @jimwbaldwin

Heya, confirming that #43622 is working correctly after installing 9.1.0rc4