target / data-validator

A tool to validate data, built around Apache Spark.
Other
101 stars 34 forks source link

Range check configuration at debug log level #56

Closed phpisciuneri closed 3 years ago

phpisciuneri commented 3 years ago

The range check configuration should be a debug log level to make it consistent with how other row-based tests are logged.

---
numKeyCols: 2
numErrorsToReport: 1
detailedErrors: true

outputs:
  - filename: report.json
    append: false

tables:
  - db: census_income
    table: adult
    checks:
      - type: rangeCheck
        column: age
        minValue: 40
        maxValue: 50

yields:

21/01/14 14:34:43 INFO Main$: Logging configured!
21/01/14 14:34:43 INFO Main$: Data Validator
21/01/14 14:34:43 INFO ConfigParser$: Parsing `issue.yaml`
21/01/14 14:34:43 INFO ConfigParser$: Attempting to load `issue.yaml` from file system
21/01/14 14:34:43 INFO RangeCheck$$anonfun$fromJson$1: RangeCheckJson: {
  "type" : "rangeCheck",
  "column" : "age",
  "minValue" : 4e1,
  "maxValue" : 5e1
}
...
...