Store/Compactor infinitely cache meta.json files

[GiedriusS]

Thanos Compact/Store and possibly other components that use meta fetcher cache meta.json files locally - on disk and in memory. If meta.json files change in remote object storage then it's hard to change them in Compact/Store/etc. because the fetcher cache has no TTL: https://github.com/thanos-io/thanos/blob/main/pkg/block/fetcher.go#L248-L253. Perhaps a TTL could be added?

[yeya24]

What's the usecase of changing the meta.json file content? Usually we consider they are immutable and create new blocks instead.

[fpetkovski]

Adding new fields often needs cache invalidation, but maybe we can do that somehow through Thanos itself.