apache / rocketmq-externals

Mirror of Apache RocketMQ (Incubating)
4.55k stars 3.07k forks source link

[RIP-66] Support KV(Rocksdb) Storage #921

Closed fujian-zfj closed 11 months ago

fujian-zfj commented 11 months ago

The issue tracker is ONLY used for bug report and feature request.

FEATURE REQUEST

We will introduce kv(rocksdb) storage to solve the possible Full GC caused by large object memory allocation when metadata such as topic is serialized, and the performance degradation caused by random writing of a large number of small ConsumeQueue files in a million-topic scenario

This proposal mainly optimizes and solves the existing perfermance problems in the million-topic scenario from two levels:

1.Metadata like topic, subscription, consumerOffset implement kv storage 2.ConsumeQueue index file implements kv storage