senecajs / seneca-beanstalk-transport

Seneca micro-services message transport over beanstalkd queues.
MIT License
3 stars 8 forks source link

[Question] How to queue jobs without waiting for a response #3

Closed imjoshholloway closed 9 years ago

imjoshholloway commented 9 years ago

Continuation of this Twitter conversation. Basically, I have a service that will process records from beanstalk:

services/intercom.srv.js:

'use strict';

var seneca = require('seneca')()
  .use('beanstalk-transport')
  .use('../lib/intercom')
  .listen({
    type: 'beanstalk',
    host: SRV_HOST,
    pin: 'role:intercom,cmd:*'
  });

lib/intercom.js:

'use strict';

module.exports = function (options) {
  var seneca = this;
  var plugin = 'intercom';

  seneca.add({role:'intercom', cmd: 'add-user', function (payload, done) {
    // do stuff and then call `done()`
  });
};

The queue is populated by the following file which is a separate process running every minute or so:

app.js:

'use strict';

var seneca = require('seneca')()
  .use('beanstalk-transport')
  .client({
    type: 'beanstalk',
    host: 'docker.app',
    pin: 'role:intercom,cmd:*'
  })
  .on('ready', start);

function start() {
  // do stuff
  seneca.act('role:intercom,cmd:add-user', data);
}

After a short period of time the following error is output in the logs of the app.js process:

seneca: Action undefined failed: [TIMEOUT].act_execute
{id=5ye1maeu9zb1/f4t6kzn1b9zx,gate=false,ungate=false,desc=role:intercom,cmd:*,plugin={name=undefined,tag=undefined},time={start=1431510551723,end=1431510585061},message=[TIMEOUT],pattern=undefined,instance=Seneca/0.6.1/l0572x189sz8/1431510548581/21139/-}
Error: [TIMEOUT]
    at null._onTimeout (/Users/imjoshholloway/code/projects/importer/node_modules/seneca/node_modules/gate-executor/gate-executor.js:95:21)
    at Timer.listOnTimeout (timers.js:110:15) 

I basically need the app.js process to simply add the jobs to the queue and forget about them. - Is this possible?

imjoshholloway commented 9 years ago

A bit more information: If I start the intercom.srv.js process after the errors are logged the jobs in the queue are processed but not removed from the queue.

maxired commented 9 years ago

If I understand correctly the question is, Is there a fire and forget mechanism in Seneca ? Not from what I know, but @rjrodger or @mcdonnelldean should be able to give a better answer.

mcdonnelldean commented 9 years ago

Fire and forget is support. I'm tight on time right now but I will see can I get @geek to look over your issue. I'd say more likely something silly is causing the timeout.

geek commented 9 years ago

@imjoshholloway you can increase the timeout to a large number (https://github.com/rjrodger/seneca/blob/master/seneca.js#L62).

This is a few months old now, are you still experiencing the issue?

maxired commented 9 years ago

@geek , wouldn't that lead to a kind of memory leak ?

imjoshholloway commented 9 years ago

@geek I'm no longer using seneca (well node) so no I'm not experiencing this anymore.

mcdonnelldean commented 9 years ago

@imjoshholloway Sorry to hear that but good look in whatever you are doing now :D