awslabs / python-deequ

Python API for Deequ
Apache License 2.0
669 stars 131 forks source link

satisfies test fails if assertion is not provided #182

Open poolis opened 6 months ago

poolis commented 6 months ago

Describe the bug satisfies test fails if assertion is not provided despite there being a default value for the argument.

I believe the reference satisfies$default$2 should be satisfies$default$3 but I need to test that.

To Reproduce

from pydeequ.checks import *
from pydeequ.verification import *

check = Check(spark, CheckLevel.Warning, "test satisfies")

checkResult = VerificationSuite(spark)
.onData(df)
.addCheck(
check
.satisfies("r in ('123','456','789')","r cannot contain any values excpet"))
.run()

Error:

py4j.Py4JException: Method satisfies$default$2([]) does not exist

--> 542 else getattr(self._Check, "satisfies$default$2")()
543 hint = self._jvm.scala.Option.apply(hint)
544 self._Check = self._Check.satisfies(columnCondition, constraintName, assertion_func, hint)

Expected behavior Test runs without error

Additional context Spark 3.3, pydeequ==1.1.1

poolis commented 6 months ago

I have verified that satisfies$default$3 resolves the issue, I will submit a PR.