Open evenyag opened 5 months ago
There are several issues related to the slow scan speed
last_yield_pk_index
only stores matched keys so we have to re-prune unmatched keysIs this issue the cause to https://github.com/orgs/GreptimeTeam/discussions/3461?
Is this issue the cause to https://github.com/orgs/GreptimeTeam/discussions/3461?
I think so.
The old TimeSeriesMemtable
always outperforms the new memtable when the number of time series is small. We still need to keep the old memtable and maybe use it as the default memtable. The new memtable is mainly optimized for the metric engine.
I think we can support per-table memtable option and enable the new memtable in the metric engine.
MetricsMemtable
I think we can close this issue currently. @evenyag
What type of bug is this?
Performance issue
What subsystems are affected?
Storage Engine
Minimal reproduce step
Use TSBS to generate some points with 4k hosts
Load it into the db
Enable debug log of the storage engine
Select some data
What did you expect to see?
The memtable scan time should be close to the old memtable
What did you see instead?
The memtable scan time is quite high
What operating system did you use?
NA
What version of GreptimeDB did you use?
0.7
Relevant log output and stack trace