search

apache / celeborn

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
https://celeborn.apache.org/
Apache License 2.0
881 stars 360 forks source link

[CELEBORN-1575] TimeSlidingHub should remove expire node when reading #2702

Closed leixm closed 2 months ago

leixm commented 2 months ago

What changes were proposed in this pull request?

TimeSlidingHub remove expire node when reading.

Why are the changes needed?

Metrics UserProduceSpeed is not correct.

You can see that if a user no longer writes data, UserProduceSpeed ​​still retains the last value until user expires. image

Does this PR introduce any user-facing change?

No.

How was this patch tested?

UTs.

leixm commented 2 months ago

@AngersZhuuuu @boneanxs @RexXiong @SteNicholas PTAL.

SteNicholas commented 2 months ago

@leixm, thanks for contribution. Merged to main(v0.6.0), branch-0.5(v0.5.2) and branch-0.4(v0.4.3).