apache / helix

Mirror of Apache Helix
Apache License 2.0
462 stars 224 forks source link

ZkBucketDataAccessor GC never occurs in high frequency write scenarios #2872

Closed GrantPSpencer closed 3 weeks ago

GrantPSpencer commented 1 month ago

Describe the bug

When a write occurs, ZkBucketDataAccessor schedules a GC task to execute 60s (default ttl) from when the task was submitted. If there already exists a GC task, updateGCTimer attempts to cancel it reschedule it for 60s from then.

With this current logic, if the time between subsequent writes is never slower than the ttl (60s), then a GC task will never be executed - the task will constantly be scheduled and then cancelled.

To Reproduce

Call compressed compressedBucketWrite every ttl / 2 seconds and observe that nodes are never cleaned up.

Expected behavior

GC should occur at some point.

Additional context

Add any other context about the problem here.