Open frendhisaido opened 1 month ago
With the information you gave us, I think that the issue could be that the worker is not being correctly connected to the Redis instance. Since you are not attaching an error listener to the worker you may be missing the connection error, you can try with:
worker.on("error", (err) => {
console.log(err);
});
Other than that I do not see any issue in the code. You can also always connect to the Redis instance using redis-cli and issue the monitor command, to see if the worker is indeed trying to fetch any jobs.
$ redis-cli monitor
I'm facing an issue with a BullMQ setup across Cloud Run and GKE, and I'm hoping someone here might have encountered a similar scenario. My Setup:
The Problem:
While the proxy successfully accepts incoming jobs (confirmed via redis-cli and debugging via QueueEvents), the BullMQ worker doesn't seem to pick them up. This happens both when the worker runs as a GKE workload and when I run it manually via bun worker.js. Has anyone successfully deployed a similar setup with BullMQ-proxy on Cloud Run and a BullMQ worker in GKE? Any insights or potential pitfalls to watch out for would be greatly appreciated!
The worker code
The strange thing is that we've successfully run this same setup in easypanel, where the bullmq-worker, bullmq-proxy, and Redis all run on the same host—almost like managing them with a single docker-compose.yml file. In that environment, the same worker code consistently picks up jobs without a hitch. We even scaled up to two worker replicas in easypanel, and everything continued to function smoothly.
We could continue using easypanel, but it's not a viable option for the long run. We're just so curious to understand why we're facing this issue with Cloud Run and GKE. I'm hoping it's just a silly configuration oversight on my part, and we can get this resolved quickly and move on with our lives! 😄