FRosner / drunken-data-quality

Spark package for checking data quality
Apache License 2.0
222 stars 69 forks source link

Review Check.hasNumRows #134

Open Gerrrr opened 7 years ago

Gerrrr commented 7 years ago

While working on #133 I checked Check.hasNumRows type signature and was curious why it is expected: Column => Column. Wouldn't it be simpler to use Long => Boolean or something like that? Also, it'd be possible to use df.count() instead of https://github.com/FRosner/drunken-data-quality/blob/master/src/main/scala/de/frosner/ddq/constraints/NumberOfRowsConstraint.scala#L9.