[AUDIT][SPARK-36620][SHUFFLE] Add Push Based Shuffle client side read metrics

NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs

https://nvidia.github.io/spark-rapids

Apache License 2.0

826 stars 236 forks source link

[AUDIT][SPARK-36620][SHUFFLE] Add Push Based Shuffle client side read metrics #7511

Open jbrennan333 opened 1 year ago

jbrennan333 commented 1 year ago

apache/spark@fb67ce2680 adds a bunch of new metrics for push based shuffle. Do we care about this in our shuffle code?

abellina commented 1 year ago

We do. We had to copy/port over the push helper and the block fetcher iterator, both classes were updated in this PR. It would make sense to "sync up" these classes.