taskqredisq: pending failed: redisq: can't find pending message

mvisonneau / gitlab-ci-pipelines-exporter

Prometheus / OpenMetrics exporter for GitLab CI pipelines insights

Apache License 2.0

1.29k stars 242 forks source link

taskqredisq: pending failed: redisq: can't find pending message #612

Open jinvin opened 1 year ago

jinvin commented 1 year ago

Hi ,

I'm receiving this error when I try to add more projects to the running Gitlab-exporter with the helm values. For eg: from the below existing projects to projects:

name: foo/bar

project:

name: foo/bar1
name: foo/bar2
name: foo/bar3

Error : taskqredisq: pending failed: redisq: can't find pending message

Any idea ?

aaronborden-rivian commented 1 year ago

I also see this error message, although we are only monitoring a single project with a lot of activity. Version: 0.5.4 via Helm chart

ubershloder commented 1 year ago

same for me

Stazis555 commented 1 year ago

restarting redis (or maybe flushing the queue) fixes it, I wish it would recover it self when it cannot get pending message. Otherwise any restart can block it completely.

ledouxpl commented 4 months ago

Same issue for me...

I get a bunch of redisq: can't find pending message id followed by context canceled.

time="2024-06-11T19:22:25Z" level=warning msg="taskq\"msg\"=\"redisq: scheduler failed\" \"error\"=\"redisq: can't find pending message id=\\\"1717709520071-0\\\" in stream=\\\"taskq:{default}:stream\\\"\" \"name\"=\"pending\""
time="2024-06-11T19:22:37Z" level=warning error="context canceled"

I am using GCP Memory Store. Restarting the instance is not something I can consider.

ledouxpl commented 4 months ago

Using the exact same configuration and topology but with an in-cluster redis instance made it work.