sparkutils / quality

A Quality Spark DQ Library
https://sparkutils.github.io/quality/
Apache License 2.0
4 stars 2 forks source link

add generic tuple based metarule #44

Open chris-twiner opened 1 year ago

chris-twiner commented 1 year ago

Metarules currently use filters against fields in a dataset then generate rules, which is good for generic nullable style tests but not for much else.

What should be added is a list of tuples with a lambda. The arity of the lambda is used to enforce the list has the correct type. Same ordering guarantee is needed, obviously it does so over the tuple.

chris-twiner commented 1 year ago

low prio as it doens't add much over current rule setups