dbt-labs / dbt-spark

dbt-spark contains all of the code enabling dbt to work with Apache Spark and Databricks
https://getdbt.com
Apache License 2.0
405 stars 227 forks source link

[ADAP-478] Remove Databricks specific tests #732

Open emmyoop opened 1 year ago

emmyoop commented 1 year ago

High Level Task

Remove the Databricks profiles from dbt-spark integration tests.

Acceptance Criteria

Details

This is a bit complex since tests are tagged to be skipped, not to be run. The tests that need to be fully removed are the ones that have the following decorator:

@pytest.mark.skip_profile("spark_session", "apache_spark")

If the decorator contains databricks_sql_endpoint, databricks_sql_endpoint or databricks_http_cluster the decorator just need to be modified.

conftest.py also needs to be modified to remove the databricks profiles.

Additional Notes

Since we will be removing the Databricks profiles in the GitHub Action, this will be dead code but the tests will continue to work.