mpneuried / rsmq-worker

Helper to simply implement a worker around RSMQ ( Redis Simple Message Queue )
MIT License
117 stars 24 forks source link

OfflineQueue - configuration to turn on and off #40

Open youngamichael opened 6 years ago

youngamichael commented 6 years ago

First off, I like this library. I wish there was a configuration for the offline queue feature to shut it off. When you call send and redis is down it pushes the message into the offline queue. The problem I see with that is redis could be down for a while and you have a potential to back up fairly significantly. It isn't written to disk so if the program would crash those messages would be lost. In some cases the offline queue maybe very useful but I wish there was a configuration option to just callback with an error so further steps could be taken upstream.

rogerweb commented 6 years ago

I think you missed the answer, so here it goes:

https://github.com/smrchy/rsmq/issues/80#issuecomment-405271826

Regards,

Roger

youngamichael commented 6 years ago

I may be mistaken but it appears that rsmq-worker implements it's own offline_queue. I couldn't see anywhere in the node-redis library where the enable_offline_queue setting effects the queue.connected property.

Code from my node_modules folder.

  RSMQWorker.prototype.send = function() {
      var args, cb, delay, msg;
      msg = arguments[0], args = 2 <= arguments.length ? slice.call(arguments, 1) : [];
      delay = args[0], cb = args[1];
      if (_isFunction(delay)) {
        cb = delay;
        delay = null;
      }
      if (delay == null) {
        delay = this.config.defaultDelay;
      }
      if (this.queue.connected) {
        this._send(msg, delay, cb);
      } else {
        this.debug("store message during redis offline time", msg, delay);
        this.offlineQueue.push({
          msg: msg,
          delay: delay,
          cb: cb
        });
      }
      return this;
    };
rogerweb commented 6 years ago

I just ran a quick test here and, although the redis client supports the flag enable_offline_queue, it looks rsmq and/or rsmq-worker don't work if you set it to false, no matter if you set it as an option of the redis client itself or under options.options of the rsmq or under options.options of the rsmq-worker or any other way you chain the creation of these three objects.

So I'll leave it to @mpneuried and @smrchy.

Cheers,

Roger

youngamichael commented 6 years ago

Now I wonder if the offline queue feature should just be removed and just lean on the node-redis offline queue. Either way I would appreciate the ability to turn it off and just let it fail. If I get some time I might submit a PR.