Open jpds opened 1 year ago
I agree that 3m is too aggressive and can cause excessive load and costs on object storage. However, a default of 1h
might be too high since some users can have a max head block duration that's also 1h
. Maybe something like 15m
could be a good middle ground, especially because this is a default and changing it affect everyone who does not override it (which is probably most users).
15 minutes sounds like a reasonable value to me 👍
I've spent the past week experimenting with different sync values on my store APIs, and I think the ideal value for the default would be 1h3m
.
The extra 3m
is arbitrary, however the repeater at https://github.com/thanos-io/thanos/blob/v0.31.0/pkg/runutil/runutil.go#L69 has no way of taking into account the duration of the previous run. A process, started at :58
past the hour, will always run sync tasks at :58
past, which could be non-ideal when new blocks are uploaded at :00
by the shipper. Hence the extra value to nudge this forward as syncs happen.
I also found 15m
to be too frequent, especially when more store APIs are spun up.
Isn't it a configurable flag? So you can change it based on your own usecase. 15m
here seems a sane default that works for most of the usecases.
I think the store API's
--sync-block-duration
should be increased from its current3m
to1h
.New blocks are normally set to be shipped every
2h
by the sidecars, and the more recent metrics are handled by the queriers.I self host my own S3 storage layer and this is what
3m
looks like on the backend:This is
15m
:And this two store APIs set to
45m
(earlier today):