search

cortexproject / cortex

A horizontally scalable, highly available, multi-tenant, long term Prometheus.
https://cortexmetrics.io/
Apache License 2.0
5.46k stars 793 forks source link

Weird chunk flush timestamps #1955

Closed bboreham closed 4 years ago

bboreham commented 4 years ago

Seen in event sampling:

msg:add to flush queue reason:MultipleChunksInSeries ts:January 6th 2020, 07:00:12.513 userID:13,085 firstTime:January 5th 2020, 17:40:42.000 fp:40e1bc94d3ca533c series:{__name__="container_last_seen", ...}

Max-chunk-age is set at 8 hours, so why is firstTime over 13 hours before ts ?

The code that logs this takes firstTime from the series so maybe the first chunk was flushed at the correct time, is marked as flushed, but somehow is still hanging around?

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.