Inefficient compaction strategy when there are more than 2T data in a compaction group with default config

[hzxa21]

By default, the LSM tree of a compaction group can have at most 7 levels (L0 - L6) with base level (i.e. the first non-L0 level) size set to 512MB and level multiplier set to 5x.

There is an implicit assumption under this configuration:

the maximum size for the LSM = 512MB + 512MB * 5 + ... 512MB * (5^5) = ~2TB

Given that the compaction strategy is based on the maximum 2TB LSM assumption, the strategy will be inefficient and pick the wrong level when generating compaction task.

[hzxa21]

Example collected in a cluster with >2TB compaction group 2:

• version information: version.txt
• score information:

  +--------------+--------------+-------------+-------+
  | Select Level | Target Level | Type        | Score |
  +===================================================+
  | 0            | 0            | Intra       | 2717  |
  |--------------+--------------+-------------+-------|
  | 0            | 0            | Tier        | 608   |
  |--------------+--------------+-------------+-------|
  | 0            | 1            | ToBase      | 2718  |
  |--------------+--------------+-------------+-------|
  | 1            | 2            | BottomLevel | 1383  |
  |--------------+--------------+-------------+-------|
  | 2            | 3            | BottomLevel | 1393  |
  |--------------+--------------+-------------+-------|
  | 3            | 4            | BottomLevel | 501   |
  |--------------+--------------+-------------+-------|
  | 4            | 5            | BottomLevel | 118   |
  |--------------+--------------+-------------+-------|
  | 5            | 6            | BottomLevel | 100   |
  +--------------+--------------+-------------+-------+