mpneuried / rsmq-worker

Helper to simply implement a worker around RSMQ ( Redis Simple Message Queue )
MIT License
117 stars 24 forks source link

rsmq-worker does not work with a ioredis client and Sentinels #21

Open antoniodimariano opened 8 years ago

antoniodimariano commented 8 years ago

Hi to all,

I'm using ioredis to establish a connection to Redis and use it as a client to RSMQ https://github.com/smrchy/rsmq and RSMQ-worker https://github.com/mpneuried/rsmq-worker.

If I use a standalone Redis server with default settings localhost:6370, all my code work as charm and tasks are being processed as expected.

Issues appear as soon as I try to use ioredis with Sentinels option (https://github.com/luin/ioredis#sentinel)

So after shut down the standalone Redis instance, I have set up a Sentinels environments with

a master server and 2 slaves servers in my local machine. Then I connect to them with

var redis_client = new Redis({
     sentinels: [{host: "localhost", port: "16380"},{host: "localhost", port: "16381"},{host: "localhost",             port: "16382"}],
name: "redis-cluster"

}); as reported in https://github.com/luin/ioredis#sentinel. Apparently everything seems to work correctly, and

redis_client.on('connect;,function(){} says that the client has been connected to the Sentinels servers. Then according to the RSMQ documentation I can use the already existing Redis client with the RSMQWorker

var worker = new RSMQWorker("mytask", { interval: [0, 10], autostart: true, maxReceiveCount: variables.rsmq.maxReceiveCount, customExceedCheck: fnCheck, redis: redis_client, redisPrefix: "rsmq", alwaysLogErrors: true }); but unfortunately with these settings tasks stop to work. As a clue the Redis Master server receives nothing on monitor. So I started to investigate and first of all I have printed out the redis_client object, just to discover a couple of localhost connections in it.

[...] options: { sentinels: [ [Object], [Object], [Object], [Object], [Object] ], name: 'redis-cluster', port: 6379, <--------- host: 'localhost', <---------- [...] SentinelConnector { options: { sentinels: [Object], name: 'redis-cluster', port: 6379, <------ host: 'localhost', <------ family: 4, [...] Does someone knows why even if I connect using Sentinels, the redis_client json has localhost and local port values on ?

I was wondering if someone has experience with Sentinels and with these kind of issues.

Thanks in advance

REDIS CLIENT: Redis { domain: null, _events: { connect: [ [Function], [Object] ], close: { [Function: g] listener: [Function] } }, _eventsCount: 2, _maxListeners: undefined, options: { sentinels: [ [Object], [Object], [Object], [Object], [Object] ], name: 'redis-cluster', port: 6379, host: 'localhost', family: 4, connectTimeout: 3000, retryStrategy: [Function], keepAlive: 0, connectionName: null, role: 'master', sentinelRetryStrategy: [Function], password: null, db: 0, parser: null, dropBufferSupport: false, enableOfflineQueue: true, enableReadyCheck: true, autoResubscribe: true, autoResendUnfulfilledCommands: true, lazyConnect: false, keyPrefix: '', reconnectOnError: null, readOnly: false, stringNumbers: false }, scriptsSet: {}, commandQueue: { [String: '[object Object]'] _capacity: 16, _length: 1, _front: 0 }, offlineQueue: { [String: ''] _capacity: 16, _length: 0, _front: 0 }, connector: SentinelConnector { options: { sentinels: [Object], name: 'redis-cluster', port: 6379, host: 'localhost', family: 4, connectTimeout: 3000, retryStrategy: [Function], keepAlive: 0, connectionName: null, role: 'master', sentinelRetryStrategy: [Function], password: null, db: 0, parser: null, dropBufferSupport: false, enableOfflineQueue: true, enableReadyCheck: true, autoResubscribe: true, autoResendUnfulfilledCommands: true, lazyConnect: false, keyPrefix: '', reconnectOnError: null, readOnly: false, stringNumbers: false }, connecting: true, retryAttempts: 0, currentPoint: 0, sentinels: [ [Object], [Object], [Object], [Object], [Object] ], stream: Socket { _connecting: false, _hadError: false, _handle: [Object], _parent: null, _host: null, _readableState: [Object], readable: true, domain: null, _events: [Object], _eventsCount: 7, _maxListeners: undefined, _writableState: [Object], writable: true, allowHalfOpen: false, destroyed: false, bytesRead: 0, _bytesDispatched: 14, _sockname: null, _pendingData: null, _pendingEncoding: '', server: null, _server: null, _idleTimeout: -1, _idleNext: null, _idlePrev: null, _idleStart: 1244, read: [Function], _consuming: true, _peername: [Object] } }, retryAttempts: 0, status: 'connect', condition: { select: 0, auth: null, subscriber: false }, stream: Socket { _connecting: false, _hadError: false, _handle: TCP { _externalStream: {}, fd: 16, reading: true, owner: [Circular], onread: [Function: onread], onconnection: null, writeQueueSize: 0 }, _parent: null, _host: null, _readableState: ReadableState { objectMode: false, highWaterMark: 16384, buffer: [], length: 0, pipes: null, pipesCount: 0, flowing: true, ended: false, endEmitted: false, reading: true, sync: false, needReadable: true, emittedReadable: false, readableListening: false, defaultEncoding: 'utf8', ranOut: false, awaitDrain: 0, readingMore: false, decoder: null, encoding: null, resumeScheduled: false }, readable: true, domain: null, _events: { end: [Object], finish: [Function: onSocketFinish], _socketEnd: [Function: onSocketEnd], error: [Object], close: [Object], data: [Function], timeout: [Object] }, _eventsCount: 7, _maxListeners: undefined, _writableState: WritableState { objectMode: false, highWaterMark: 16384, needDrain: false, ending: false, ended: false, finished: false, decodeStrings: false, defaultEncoding: 'utf8', length: 0, writing: false, corked: 0, sync: false, bufferProcessing: false, onwrite: [Function], writecb: null, writelen: 0, bufferedRequest: null, lastBufferedRequest: null, pendingcb: 1, prefinished: false, errorEmitted: false, bufferedRequestCount: 0, corkedRequestsFree: [Object] }, writable: true, allowHalfOpen: false, destroyed: false, bytesRead: 0, _bytesDispatched: 14, _sockname: null, _pendingData: null, _pendingEncoding: '', server: null, _server: null, _idleTimeout: -1, _idleNext: null, _idlePrev: null, _idleStart: 1244, read: [Function], _consuming: true, _peername: { address: '127.0.0.1', family: 'IPv4', port: 6380 } }, replyParser: JavascriptReplyParser { name: 'javascript', buffer: , offset: 0, bigStrSize: 0, chunksSize: 0, buffers: [], type: 0, protocolError: false, offsetCache: 0, handleReply: [Function], handleNumbers: [Function], returnError: [Function], returnFatalError: [Function], returnReply: [Function] } }

mpneuried commented 8 years ago

i will have a look at it this week. But probably this is an issue inside rsmq.