Closed Kontinuation closed 1 month ago
The metrics of CometSort nodes shown in the SQL tab of Spark UI have invalid spilled bytes metrics. It shows spilled bytes as time instead of size.
Run any SQL queries involving native sort with comet enabled.
Spill size shown as size instead of time.
This problem is reproduced using the latest commit of the main branch: https://github.com/apache/datafusion-comet/commit/f64553b6b788484e3ca03ab8d7b19e51670444ee
Describe the bug
The metrics of CometSort nodes shown in the SQL tab of Spark UI have invalid spilled bytes metrics. It shows spilled bytes as time instead of size.
Steps to reproduce
Run any SQL queries involving native sort with comet enabled.
Expected behavior
Spill size shown as size instead of time.
Additional context
This problem is reproduced using the latest commit of the main branch: https://github.com/apache/datafusion-comet/commit/f64553b6b788484e3ca03ab8d7b19e51670444ee