vsivsi / meteor-job-collection

A persistent and reactive job queue for Meteor, supporting distributed workers that can run anywhere.
https://atmospherejs.com/vsivsi/job-collection
Other
388 stars 68 forks source link

workers on separate meteor server won't trigger #181

Closed danielparas closed 8 years ago

danielparas commented 8 years ago

Hi, thanks for the great package!

I've setup job collection on 'Sever A' which will act as the meteor job server. I define the collection globally and save a job and run jobCollection.startJobServer() within startup function of 'Server A'

/server/lib/collections/jobCollection.coffee

@jobCollection = new JobCollection 'job_queue',
  noCollectionSuffix:true

server/init.coffee

Meteor.startup ->

    #Setup the job
    job = new Job(jobCollection, 'sendEmail',
      address: 'bozo@clowns.com'
      subject: 'Critical rainbow hair shortage'
      message: 'LOL; JK, KThxBye.')
    job.save()
    console.log 'jobsaved'

    #Start job server
    jobCollection.startJobServer()
  return 

Now on 'Server B', which will act as a meteor based worker, i define the job collection in a similar manner.

/server/lib/collections/jobCollection.coffee

@jobCollection = new JobCollection 'job_queue',
  noCollectionSuffix:true

And setup the worker in the startup function of the server

server/init.coffee

Meteor.startup ->
  queue = jobCollection.processJobs('sendEmail', {
    pollInterval: 50
  }, (job, callback) ->
    console.log 'im running!'
    job.done()
    callback()
    return
  )
return

'Server B' is configured to use the same MongoDB URL as 'Server A', and running a command like jobCollection.find({}).fetch() on Server B returns the jobs defined on 'Server A' correctly. All jobs returned by the latter call are in a readystate. However the jobs never seem to trigger the worker function to run. Running jobCollection.getWork('sendEmail'); on meteor shell of 'Server B' returns an empty set []

Any ideas what I'm doing wrong please? If I add jobCollection.startJobServer() to Server B's statrup function, workers trigger and complete successfully - but running jobCollection.startJobServer() on both Server A and Server B makes both servers the job server which is not correct.

Thanks in advance for any help! Dan

vsivsi commented 8 years ago

Hi, if you truly just want server B to be a client, then you need to configure its JobCollection to point to server A (as a client would) and not to the mongoDB. That is, you need server B to have an active DDP connection to Server A.

See: http://docs.meteor.com/#/full/ddp_connect

Then you can pass the DDP connection for server A to server B's JobCollection when you create it (using the connection option in the options object. See: http://docs.meteor.com/#/full/mongo_collection

danielparas commented 8 years ago

hey @vsivsi thanks for the quick reply! Ah ok so the server must still act as a normal DPP client - I was trying to bypass this to avoid having a user for the worker. Since that is what needs to be done just the same, makes more sense to use a lightweight pure node worker.

Thanks for your help - highly appreciated!