Is your feature request related to a problem? Please describe.
We recently had a requirement to provide the capability to maintain active search of our logs for 365 days. As an organization we are looking to optimize searches within the past 30 days and keep as much data in cache as possible for this window in the cache to avoid having to go to object storage to retrieve data.
From my understanding, when a query is run, the chunks that are pulled back from object storage are written to the chunks cache. We are looking to avoid the cache churn for infrequent queries that are beyond the desired duration (in this case 30 days). With this configuration we would reduce the amount of data being written tot he chunks cache and reduce the evictions for older data that might not ever be viewed again.
Describe the solution you'd like
A configuration option is provided that would allow a duration to be specified that would allow the querier to write back to the chunk cache. If the chunk falls outside of this duration, it would not write the result back to the chunk cache.
Describe alternatives you've considered
The alternative is to build a bigger caching infrastructure to handle a longer duration and absorb the older and infrequent data written to the cache
Is your feature request related to a problem? Please describe. We recently had a requirement to provide the capability to maintain active search of our logs for 365 days. As an organization we are looking to optimize searches within the past 30 days and keep as much data in cache as possible for this window in the cache to avoid having to go to object storage to retrieve data.
From my understanding, when a query is run, the chunks that are pulled back from object storage are written to the chunks cache. We are looking to avoid the cache churn for infrequent queries that are beyond the desired duration (in this case 30 days). With this configuration we would reduce the amount of data being written tot he chunks cache and reduce the evictions for older data that might not ever be viewed again.
Describe the solution you'd like A configuration option is provided that would allow a duration to be specified that would allow the querier to write back to the chunk cache. If the chunk falls outside of this duration, it would not write the result back to the chunk cache.
Describe alternatives you've considered The alternative is to build a bigger caching infrastructure to handle a longer duration and absorb the older and infrequent data written to the cache