[Feature] Add spark support - Githubissues

calogica / dbt-expectations

Port(ish) of Great Expectations to dbt test macros

https://calogica.github.io/dbt-expectations/

Apache License 2.0

989 stars 120 forks source link

[Feature] Add spark support #279

Closed clausherther closed 9 months ago

clausherther commented 10 months ago

Issue this PR Addresses/Closes

Closes #280

Summary of Changes

This PR updates test macros and supporting integration tests, as well as CircleCI configs to support apache-spark.

Also updates dependency on dbt-date to latest version (0.9.x)

Regex support is currently experimental in integration testing (and set to warn only), as support for raw regex strings in SparkSQL is unclear.

Why Do We Need These Changes

Shim packages like spark-utils do not implement support for dbt-expectations.