search

apache / celeborn

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
https://celeborn.apache.org/
Apache License 2.0
886 stars 359 forks source link

[CELEBORN-1522] Fix applicationId extraction from shuffle key #2645

Closed turboFei closed 3 months ago

turboFei commented 3 months ago

What changes were proposed in this pull request?

Fix applicationId extraction from shuffle key.

Why are the changes needed?

For spark on k8s, the applicationId might be spark-da4571bd2cbf491c892cbd4de40fc918.

Due the application extraction is not correct, the result of API /api/v1/applications/top_disk_usages is not correct.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Not needed, just leverage existing method.

turboFei commented 3 months ago

thanks, merged to main, branch-0.5, branch-0.4