search

kubeflow / spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
Apache License 2.0
2.73k stars 1.36k forks source link

Why the least result size sent to sender is 10952? #1687

Open dongyunpeng-sjtu opened 1 year ago

dongyunpeng-sjtu commented 1 year ago

I run a spark-on-k8s-operator with remote shuffle service. but some tasks' shuffle read size is 0B but after Finishing this kind of task. 10952 bytes result sent to driver Is there any data?

dongyunpeng-sjtu commented 1 year ago

image

dongyunpeng-sjtu commented 1 year ago

image

I wonder if TID 950 has not read the data, or if it has read the data by means of process_locality, but only a small part

dongyunpeng-sjtu commented 1 year ago

image

github-actions[bot] commented 2 weeks ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.