This PR introduces a new file format for storing WAL segments directly on object storage. This is part of an experiment to replace the ring replication factor and delegate this responsibility to the object storage.
I had some hesitationw with the format, particularly with tenants, may be storing an index per tenant is better, in case each tenant has big indexes. We will learn more once we run things.
What this PR does / why we need it:
This PR introduces a new file format for storing WAL segments directly on object storage. This is part of an experiment to replace the ring replication factor and delegate this responsibility to the object storage.
The format is documented at https://github.com/grafana/loki/blob/32b1d2c4acf3955801f7be9a0a43bef396e15507/pkg/storage/wal/README.md
I had some hesitationw with the format, particularly with tenants, may be storing an index per tenant is better, in case each tenant has big indexes. We will learn more once we run things.
see https://github.com/grafana/loki-private/issues/1001