mrpowers-io / spark-fast-tests

Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)
https://mrpowers-io.github.io/spark-fast-tests/
MIT License
436 stars 77 forks source link

[Performance Improvement] default to row equal implementation if tolerance is 0 #124

Closed zeotuan closed 2 months ago

zeotuan commented 6 months ago

using default equals implementation from Row should be much more performance. We can improve Dataframe equality assertion speed if strict comparison is required.

SemyonSinchenko commented 3 months ago

@zeotuan At the moment strict equality is checked first using Row.equals.