risingwavelabs / risingwave

Best-in-class stream processing, analytics, and management. Perform continuous analytics, or build event-driven applications, real-time ETL pipelines, and feature stores in minutes. Unified streaming and batch. PostgreSQL compatible.
https://go.risingwave.com/slack
Apache License 2.0
7.06k stars 581 forks source link

feat(streaming): decouple memory manager tick interval with barrier #19494

Closed MrCroxx closed 1 day ago

MrCroxx commented 1 day ago

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Previously, without sequence-based operator cache eviction, the memory manager needed to wait for barrier completions to update eviction info. However, it doesn't need that now.

We can decouple the memory manager tick interval with the barrier interval, to achieve more efficient operator cache eviction even when the barrier interval is high.

close #19403

Checklist

Documentation

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.