matrixorigin / matrixone

Hyperconverged cloud-edge native database
https://docs.matrixorigin.cn/en
Apache License 2.0
1.79k stars 276 forks source link

[Performance]: tune ap cache policy #12876

Closed badboynt1 closed 3 weeks ago

badboynt1 commented 11 months ago

Is there an existing issue for performance?

Environment

- Version or commit-id (e.g. v0.1.0 or 8b23a93):
- Hardware parameters:
- OS type:
- Others:

Details of Performance

目前的cache 策略有些问题,需要做一些调整。 根据scan的block数量将scan分成三类,小于100个block为small scan,大于某个阈值(待测试决定)为large scan。中间的为normal scan。 针对三种不同类型的scan,采用不同的prefetch,memcache policy

Additional information

badboynt1 commented 11 months ago

image

image

daviszhen commented 11 months ago

1,cn内部mem cache policy. 2,涉及filservice,存储 。 3,增加daily 截图。 4,通过revert 别人的commit,二分验证的。原因明确。 5,目标:跟之前的性能一致。顺带省内存。