search

Automattic / kue

Kue is a priority job queue backed by redis, built for node.js.
http://automattic.github.io/kue
MIT License
9.45k stars 862 forks source link

processing a queue in cluster mode fails #1227

Closed manasa-ei closed 5 years ago

manasa-ei commented 5 years ago

Try to process a queue using ioredis in cluster it fails with too many cluster redirections: sample code is as below: var Redis = require('ioredis'); const kue = require('kue'); const queue = kue.createQueue({ redis: { createClientFactory: function () { return new Redis.Cluster([{ port: 6380, host: '127.0.0.1' }, { port: 6381, host: '127.0.0.1' }, { port: 6382, host: '127.0.0.1' }, { port: 6383, host: '127.0.0.1' }, { port: 6384, host: '127.0.0.1' }, { port: 6385, host: '127.0.0.1' }], { maxRedirections: 1 }); } } }); queue.process('email', (job, done) => { const payload = job.data; console.log('jkskfgnskgnsdfkgn', payload) done() });

queue.process('testemail', (job, done) => { const payload = job.data; console.log('jkskfgnskgnsdfkgn', payload) done() });

Error is: /Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/standard-as-callback/built/index.js:7 throw e; ^

Error: Too many Cluster redirections. Last error: ReplyError: MOVED 11958 127.0.0.1:6382 at Cluster.handleError (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/ioredis/built/cluster/index.js:533:38) at Command.command.reject (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/ioredis/built/cluster/index.js:412:23) at Redis.exports.returnError (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/ioredis/built/redis/parser.js:61:26) at JavascriptRedisParser.returnError (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/ioredis/built/redis/parser.js:20:19) at JavascriptRedisParser.execute (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/redis-parser/lib/parser.js:542:14) at Socket. (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/ioredis/built/redis/event_handler.js:107:26) at Socket.emit (events.js:182:13) at addChunk (_stream_readable.js:283:12) at readableAddChunk (_stream_readable.js:264:11) at Socket.Readable.push (_stream_readable.js:219:10) Emitted 'error' event at: at Worker. (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/kue/lib/kue.js:342:12) at Worker.emit (events.js:182:13) at Worker.error (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/kue/lib/queue/worker.js:104:8) at /Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/kue/lib/queue/worker.js:85:20 at /Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/kue/lib/queue/worker.js:279:14 at tryCatcher (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/standard-as-callback/built/utils.js:11:23) at promise.then (/Users/manasamishra/xlabprojects/manas-fork/redis-cluster-ioredis/node_modules/standard-as-callback/built/index.js:31:51) at process._tickCallback (internal/process/next_tick.js:68:7)

command which fails is: command: { name: 'blpop', args: [ '{q}:testemail:jobs', '0' ] } }

Note: I have made maxRedirections to 1 so as to get the error at the earliest, otherwise it takes long time as default is 16.