Closed Lucindo06 closed 7 years ago
@Gerrrr any thoughts? What was the reason why we didn't implement?
Originally I did not figure how to convert pyspark.sql.Column
into scala's column. It is implemented in https://github.com/FRosner/drunken-data-quality/pull/118 now.
Column instance with the python version is not working. When I try to execute this code :
%pyspark from pyddq.core import Check from pyddq.reporters import ZeppelinReporter
df = sqlContext.createDataFrame([(1, "a"), (1, None), (3, "c")]) check = Check(df) reporter = ZeppelinReporter(z) check.satisfies(df._1 > 0).run([reporter])
I get : TypeError: 'Column' object is not callable