search

metrico / qryn

Polyglot Observability Stack. Lightweight & Drop-in compatible with Loki, Prometheus, Tempo, Pyroscope, Opentelemetry, Datadog & more! WASM powered ⭐️ Star to Support
https://qryn.dev
GNU Affero General Public License v3.0
1.05k stars 63 forks source link

Error "Bad Request: invalid request" with large body request to qryn #487

Open mk-raven opened 3 months ago

mk-raven commented 3 months ago

When doing a big query request in loki api by all values in labels, there was error massage:

{"level":50,"time":1711468950014,"pid":19,"hostname":"qryn-read-545cb58978-k654s","name":"qryn","reqId":"req-2vj","err":"invalid request\nBad Request: invalid request\n at module.exports.transpile (/app/parser/transpiler.js:141:11)\n at scanFingerprints (/app/lib/db/clickhouse.js:248:29)\n at Object.handler (/app/lib/handlers/query_range.js:41:28)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)","msg":"invalid request"} In request about 3250 characters and starts by: req":{"method":"GET","url":"/loki/api/v1/query_range?direction=backward&end=1711468949462000000&limit=3000 ...

Its could be problem with limits body size or something else..

lmangani commented 3 months ago

Thanks for the report! Looks like any other invalid request so if you're positive it was not somehow truncated we should investigate. Since this was a query, could you show us the query causing this error or does it contain something fancy?

deathalt commented 3 months ago

this happens on the log dashboard when the All variable is added. the query looks like this

{k8s_cluster_name="cluster1", k8s_namespace_name=~"service1|service2|service3|service|service4|service5|service6|service7|service8|service9|service10|service11|service11|service12|service13|service14|service15|service15|service16|service17|service18|service19|service20|service21|service22|service23|service24service|service25|service26|service27|service28", level="WARN"} |= "" | json body="body"

If there are fewer services, the query works without errors.

akvlad commented 2 months ago

@deathalt @mk-raven should be fixed in 3.2.18