Out of bound error in /api/v1/query_range

[thmzlt]

Some times Grafana metrics query requests start failing with "out of bounds" errors. Restarting the qyrn process seems to helmp. Here are the qryn process logs around the requests:

{"level":30,"time":1712493563160,"pid":19,"hostname":"qryn-774c84c7c-9hwvk","name":"qryn","reqId":"req-157","req":{"method":"POST","url":"/api/v1/query_range","hostname":"qryn.qryn-read:3100","remoteAddress":"10.0.0.102","remotePort":50000},"msg":"incoming request"}
{}
RangeError: offset is out of bounds
    at Uint8Array.set (<anonymous>)
    at Ctx.write (/app/wasm_parts/main.js:216:53)
    at pql (/app/wasm_parts/main.js:156:9)
    at module.exports.pqlRangeQuery (/app/wasm_parts/main.js:63:16)
    at module.exports.rangeQuery (/app/promql/index.js:21:29)
    at Object.handler (/app/lib/handlers/prom_query_range.js:22:26)
    at routeOptions.handler (/app/node_modules/@fastify/websocket/index.js:150:24)
    at preHandlerCallback (/app/node_modules/fastify/lib/handleRequest.js:137:37)
    at validationCompleted (/app/node_modules/fastify/lib/handleRequest.js:121:5)
    at preValidationCallback (/app/node_modules/fastify/lib/handleRequest.js:98:5)
{"level":30,"time":1712493563161,"pid":19,"hostname":"qryn-774c84c7c-9hwvk","name":"qryn","reqId":"req-157","res":{"statusCode":500},"responseTime":0.5298900604248047,"msg":"request completed"}

Running qryn containerized with image ghcr.io/metrico/qryn:3.2.17 in Kubernetes.

[lmangani]

Thanks for the report! Could you confirm if the issue persists in 3.2.19?

[thmzlt]

Okay, will run 3.2.19 for a week and report back.

[thmzlt]

Still getting the out of bounds in 3.2.19.

[thmzlt]

@lmangani anything else I can share to help debug this?

[lmangani]

@thmzlt could you kindly retest using the latest version and provide feedback?