Closed mbutrovich closed 2 months ago
N/A.
We want deeper insights to query performance when Spark to Arrow conversions occur.
Added a new metric to CometSparkToColumnarExec and wrapped the calls to the batch iterator's next function with timing functions.
CometSparkToColumnarExec
next
Extended an existing test that exercises SparkToColumnar to check for metrics.
Which issue does this PR close?
N/A.
Rationale for this change
We want deeper insights to query performance when Spark to Arrow conversions occur.
What changes are included in this PR?
Added a new metric to
CometSparkToColumnarExecand wrapped the calls to the batch iterator'snextfunction with timing functions.How are these changes tested?
Extended an existing test that exercises SparkToColumnar to check for metrics.