Can I disable these logs?

[salarali]

Is your feature request related to a problem? Please describe.

level=info ts=2024-03-05T21:33:16.993842816Z caller=metrics.go:159 component=querier org_id=fake traceID=0bf5202db7b6e5b9 latency=fast query="<<query>>" query_hash=1504962168 query_type=metric range_type=range length=29m0s start_delta=8h33m16.99383682s end_delta=8h4m16.993836963s step=1m0s duration=1.896618379s status=200 limit=100 returned_lines=0 throughput=256MB total_bytes=486MB total_bytes_structured_metadata=0B lines_per_second=403993 total_lines=766221 post_filter_lines=766221 total_entries=30 store_chunks_download_time=615.758114ms queue_time=1.450908339s splits=0 shards=0 cache_chunk_req=153 cache_chunk_hit=0 cache_chunk_bytes_stored=149814855 cache_chunk_bytes_fetched=0 cache_chunk_download_time=20.299832ms cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_stats_results_req=0 cache_stats_results_hit=0 cache_stats_results_download_time=0s cache_result_req=0 cache_result_hit=0 cache_result_download_time=0s source=logvolhist

These logs get printed by the querier on basically every request ( not sure about that, but looks like it). Is there a way to disable these? What are they being used for? It basically adds a lot of spam to the logs and make it harder to see error and other useful logs.

Describe the solution you'd like

Maybe move these logs to debug or add a flag using which they can be toggled.

[saule1508]

those logs are very valuable for performance and troubleshooting. If you need help to analyze those logs, head on to the loki blog where there is a very good discussion on loki performance and how those logs help. https://grafana.com/blog/2023/12/28/the-concise-guide-to-loki-how-to-get-the-most-out-of-your-query-performance/

[ptr1120]

I would also like to know how to disable these kinds of metrics logs. If they are valuable for troubleshooting, wouldn't it be better to use a debug level for these kinds of messages?

[Barsonax]

@saule1508 I think its clear that these logs can be useful but I don't want them to always fill up the logs when iam not interested in them so how to disable them?

[arusa]

I also want to be able to disable those logs. They can even create problems when searching for patterns in /var/log/syslog and it's always writing those patterns into the same log because of the query.

[nferch]

I also find these logs a bit spammy. My instance gets <.5qps yet > 15k loglines per hour. Each query seems to generate several loglines with very little difference. Setting the loglevel to warn seems a bit extreme, as they'd be useful for observability and troubleshooting, but I'd expect at most just a few lines per query.

level=info ts=2024-08-15T03:24:40.94473648Z caller=engine.go:220 component=querier org_id=fake msg="executing query" type=range quer
y="{job=\"nginx\"}" length=28.6703497s step=1s query_hash=1503951843
level=info ts=2024-08-15T03:24:40.945272838Z caller=engine.go:220 component=querier org_id=fake msg="executing query" type=range que
ry="{job=\"nginx\"}" length=28.6703497s step=1s query_hash=1503951843                                                               level=info ts=2024-08-15T03:24:40.945405553Z caller=engine.go:220 component=querier org_id=fake msg="executing query" type=range query="{job=\"nginx\"}" length=28.6703497s step=1s query_hash=1503951843
level=info ts=2024-08-15T03:24:40.945563513Z caller=engine.go:220 component=querier org_id=fake msg="executing query" type=range query="{job=\"nginx\"}" length=28.6703497s step=1s query_hash=1503951843                                                               level=info ts=2024-08-15T03:24:40.945582958Z caller=engine.go:220 component=querier org_id=fake msg="executing query" type=range que
ry="{job=\"nginx\"}" length=28.6703497s step=1s query_hash=1503951843
level=info ts=2024-08-15T03:24:40.945665184Z caller=engine.go:220 component=querier org_id=fake msg="executing query" type=range que
ry="{job=\"nginx\"}" length=28.6703497s step=1s query_hash=1503951843
level=info ts=2024-08-15T03:24:40.950114256Z caller=metrics.go:152 component=querier org_id=fake latency=fast query="{job=\"nginx\"}
" query_hash=1503951843 query_type=limited range_type=range length=28.6703497s start_delta=38.633971865s end_delta=9.963622451s step
=1s duration=4.407968ms status=200 limit=100 returned_lines=0 throughput=0B total_bytes=0B lines_per_second=0 total_lines=0 total_en
tries=0 store_chunks_download_time=0s queue_time=2.52536ms splits=0 shards=0 cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_s
tored=0 cache_chunk_bytes_fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s ca
che_result_req=0 cache_result_hit=0 cache_result_download_time=0s
level=info ts=2024-08-15T03:24:40.950153057Z caller=metrics.go:152 component=querier org_id=fake latency=fast query="{job=\"nginx\"}
" query_hash=1503951843 query_type=limited range_type=range length=28.6703497s start_delta=38.63401367s end_delta=9.963664221s step=
1s duration=4.394018ms status=200 limit=100 returned_lines=0 throughput=0B total_bytes=0B lines_per_second=0 total_lines=0 total_entries=0 store_chunks_download_time=0s queue_time=3.708572ms splits=0 shards=0 cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes_fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_result_req=0 cache_result_hit=0 cache_result_download_time=0s                                                                   level=info ts=2024-08-15T03:24:40.950564854Z caller=metrics.go:152 component=querier org_id=fake latency=fast query="{job=\"nginx\"}" query_hash=1503951843 query_type=limited range_type=range length=28.6703497s start_delta=38.634412684s end_delta=9.964063223s step=1s duration=5.638329ms status=200 limit=100 returned_lines=0 throughput=0B total_bytes=0B lines_per_second=0 total_lines=0 total_entries=0 store_chunks_download_time=0s queue_time=3.1157ms splits=0 shards=0 cache_chunk_req=0 cache_chunk_hit=0 cache_chunk_bytes_stored=0 cache_chunk_bytes_fetched=0 cache_chunk_download_time=0s cache_index_req=0 cache_index_hit=0 cache_index_download_time=0s cache_result_req=0 cache_result_hit=0 cache_result_download_time=0s