CoxAutomotiveDataSolutions / waimak

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.
Apache License 2.0
75 stars 16 forks source link

Changed delimiter for Deequ genericSQLCheck #98

Closed vavison closed 4 years ago

vavison commented 4 years ago

Description

Delimiter for the Deequ genericSQLCheck has been changed from a comma to a semicolon to allow for checks which include commas (such as col_a in ('abc', 'def'))

This is a breaking change as existing comma-delimited config will no longer work.

Fixes #97

Type of change

Please delete options that are not relevant.

How Has This Been Tested?

Additional unit test covering this scenario

codecov-io commented 4 years ago

Codecov Report

Merging #98 into develop will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop      #98   +/-   ##
========================================
  Coverage    88.32%   88.32%           
========================================
  Files           74       74           
  Lines         1790     1790           
  Branches        78       78           
========================================
  Hits          1581     1581           
  Misses         209      209
Impacted Files Coverage Δ
...taquality/deequ/prefabchecks/GenericSQLCheck.scala 100% <ø> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 6e86017...45f85bc. Read the comment docs.