apache / horaedb

Apache HoraeDB (incubating) is a high-performance, distributed, cloud native time-series database.
https://horaedb.apache.org
Apache License 2.0
2.64k stars 207 forks source link

Reduce the cost of massive stale tables #1281

Closed ShiKaiWi closed 1 month ago

ShiKaiWi commented 1 year ago

Describe This Problem

In the current implementation of the storage engine, one table must occupy some resources even if the table may not be used anymore. And the number of such tables grows larger, the waste of the resources grows too, e.g. slow recover, finding the max memory usage in all the tables.

Proposal

I could come up two ways to solve this problem:

Additional Context

1278 is an example caused by this issue.

ShiKaiWi commented 1 year ago

Opening all the tables lazily is a sound method.