search

vsivsi / meteor-job

Job class for use with Meteor job-collection distributed job queue.
https://www.npmjs.org/package/meteor-job
Other
40 stars 20 forks source link

What does startJobServer do? #15

Open rijk opened 6 years ago

rijk commented 6 years ago

Quick question: what exactly does Job.startJobServer() do, and why is it necessary?

As far as I understand the package, it works as following (for me):

  1. Initialize new JobCollection on main server
  2. Add job to this collection
  3. Worker server picks this up with a DDP observer, and checks status == ready
  4. Job.processJobs() is triggered, and does the rest

In this flow, where does the Job.startJobServer() come in? The documentation does not really explain, and I tried to figure out with the source code but I cannot find a definition for the function, so it's hard to figure out what it does.

vsivsi commented 6 years ago

It is literally what it says. The job server actively "runs" in the background after it is started, and stops "running" when it is stopped. When it is not running, it does not provide new jobs to workers that request them, nor does it accept results or other operations that mutate state. Waiting jobs aren't promoted to ready (after the appropriate time has elapsed) etc. It is a way to start and stop activity on the Job Collection for maintenance or cleaning starting / stopping servers, etc.

vsivsi commented 6 years ago

You can't find the definition because this package is just the pure JS client interface. The function is implemented for the back-end only in Meteor specific code on the other project:

https://github.com/vsivsi/meteor-job-collection/blob/master/src/shared.coffee#L388-L440

rijk commented 6 years ago

Thank you for your great reply, and the pointer to the code!

Looking at the code, what makes this activate is simply toggling the internal @stopped variable, correct? This was the cause for my confusion, because in the database all the jobs were ready, but not running. What added to my confusion, is that the worker server is the one actually running the jobs, not the application server where I have to do the Job.startJobServer() call. But of course, this works as the application server has to provide work to the worker server.

Following the example worker script, I initiated an observer on the jobs collection. I assumed this would provide the jobs to the worker, which would examine the job document to figure out if the job should run or not. This is where I now see I was mistaken, as this observer merely serves as a means to trigger the Job.processJobs queue. Which in turn will still call the application server to find out what needs to be done.

Would you mind if I do a PR to add your explanation to the function documentation in the Readme?