search

CODAIT / spark-bench

Benchmark Suite for Apache Spark
https://codait.github.io/spark-bench/
Apache License 2.0
239 stars 123 forks source link

The querytime is incorrect since it is lazy evaluated #197

Open tobegit3hub opened 3 years ago

tobegit3hub commented 3 years ago

Now the source code of spark-bench uses single scala time function to evaluate the execution time.

The time of querytime is incorrect if we just call the sql function which will return right now.

image

image

tobegit3hub commented 3 years ago

Kind of similar with https://github.com/CODAIT/spark-bench/issues/161 .

eddytruyen commented 3 years ago

Querytime is the most important part here. Does this bug also affect other Sparkbench workloads?