Vertical Block Sharding

thanos-io / thanos

Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.

https://thanos.io

Apache License 2.0

13.1k stars 2.1k forks source link

Open bwplotka opened 2 years ago

bwplotka commented 2 years ago

The proposal with motivation, proposed approach is almost ready here.

Help wanted to get that to completion.

AC:

Compactor can make sure from e.g. 4 large 2d blocks to compose 4 vertically splitter 8d blocks. This is to assure we can parallize fetch from blocks on store gateway level.

roystchiang commented 2 years ago

On the Cortex size, we also made an attempt at this problem