Open Taragolis opened 5 months ago
Happy to help out where I can here. Thanks for putting this together @Taragolis!
I can take on the task or at least support it.
Raised PR for provider postgres
Raised PR for provider weaviate
Raised PR for provider AWS EKS
I am going to raise PR for Trino.
I am going to raise PR for Trino.
oops @dirrao I am sorry I raised a PR without noticing it :(
Raising PR for Snowflake, Drill and SQLite too!
Edit: Raising for JDBC, MSSQL too
The open ones in this task are
tests/system/providers/google/cloud/life_sciences/example_life_sciences.py
tests/system/providers/google/marketing_platform/example_analytics.py
tests/system/providers/google/cloud/bigquery/example_bigquery_operations.py tests/system/providers/amazon/aws/example_ecs_fargate.py
tests/system/providers/amazon/aws/example_emr_notebook_execution.py
tests/system/providers/amazon/aws/example_emr.py
I am going to raise PR for Trino.
oops @dirrao I am sorry I raised a PR without noticing it :(
No issues. I will drop mine.
Hi @Taragolis ! Thank you for opening this issue to improve system tests for Google provider :) From our team's side we can offer checking all the PR and also running them in Google env to check if they work good with new changes. I would like to ask you only update the description of this issue and ask people who will create PRs to mention me in there, so we will be able to see all new changes. Thank you!
Body
Time to time we deprecate some attributes/arguments and even entire Operators, however maintainer might forget to change it as result system tests which use as DAG/Operators example might suggest to use deprecated stuff, this should not be a case after the https://github.com/apache/airflow/pull/39468 PR
The purpose of this task is to fixing existing usage of deprecated stuff in Airflow Providers system tests This one is a special case for https://github.com/apache/airflow/issues/38642
How to check that deprecations resolved after the changes
For example you make a changes in
tests/system/providers/google/cloud/gcs/example_gcs_to_gcs.py
and want to check that it resolvedRemove reference to specific system test from that list https://github.com/apache/airflow/blob/694d3038fca7c6127feebc370255707aceaddb01/tests/always/test_example_dags.py#L50-L80
Run test case
tests/always/test_example_dags.py::test_should_be_importable
, you might specify relative path as pytest parametrise by define it inside of[]
bracketsProvider
amazon
tests/system/providers/amazon/aws/example_ecs_fargate.py
tests/system/providers/amazon/aws/example_eks_with_nodegroups.py
tests/system/providers/amazon/aws/example_emr.py
tests/system/providers/amazon/aws/example_emr_notebook_execution.py
Provider
apache.drill
tests/system/providers/apache/drill/example_drill_dag.py
Provider
dbt.cloud
tests/system/providers/dbt/cloud/example_dbt_cloud.py
Provider
docker
tests/system/providers/docker/example_docker_swarm.py
Provider
google
tests/system/providers/google/cloud/azure/example_azure_fileshare_to_gcs.py
tests/system/providers/google/cloud/bigquery/example_bigquery_operations.py
tests/system/providers/google/cloud/bigquery/example_bigquery_sensors.py
tests/system/providers/google/cloud/dataproc/example_dataproc_gke.py
tests/system/providers/google/cloud/gcs/example_gcs_sensor.py
tests/system/providers/google/cloud/gcs/example_gcs_to_gcs.py
tests/system/providers/google/cloud/kubernetes_engine/example_kubernetes_engine.py
tests/system/providers/google/cloud/kubernetes_engine/example_kubernetes_engine_async.py
tests/system/providers/google/cloud/kubernetes_engine/example_kubernetes_engine_job.py
tests/system/providers/google/cloud/kubernetes_engine/example_kubernetes_engine_kueue.py
tests/system/providers/google/cloud/kubernetes_engine/example_kubernetes_engine_resource.py
tests/system/providers/google/cloud/life_sciences/example_life_sciences.py
tests/system/providers/google/marketing_platform/example_analytics.py
Provider
jdbc
tests/system/providers/jdbc/example_jdbc_queries.py
Provider
microsoft.mssql
tests/system/providers/microsoft/mssql/example_mssql.py
Provider
mysql
tests/system/providers/mysql/example_mysql.py
Provider
postgres
tests/system/providers/postgres/example_postgres.py
Provider
snowflake
tests/system/providers/snowflake/example_snowflake.py
Provider
sqlite
tests/system/providers/sqlite/example_sqlite.py
Provider
trino
tests/system/providers/trino/example_trino.py
Provider
weaviate
tests/system/providers/weaviate/example_weaviate_cohere.py
tests/system/providers/weaviate/example_weaviate_openai.py
tests/system/providers/weaviate/example_weaviate_operator.py
Committer