deequ not working on databricks 10.4 runtime

awslabs / deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

Apache License 2.0

3.26k stars 536 forks source link

deequ not working on databricks 10.4 runtime #428

Open hardiktalati opened 2 years ago

hardiktalati commented 2 years ago

i am trying to upgrade from 9.1 to 10.4 runtime and its throwing an issue

mfridrikhson-tp commented 2 years ago

For anyone wondering - it is a conflict between breeze dependency versions - DBR 7.3 and later use 1.x which conflicts with Deequ's 0.13.2. A version bump should fix it