Is there a right way to pass processed data from a job to its descendant?

[ThadeuLuz]

The title says it all. Right now I'm doing this on the antecedent:

Jobs.update(job._doc._id, { $set: {
  "data.stuffFound": 'stuff'
}});

and then in the descendant

Jobs.findOne(job._doc.resolved[0]).data.stuffFound

basically using it as a regular collection. It is working so far, but the fact that the docs says data can't be changed after job is created makes me wonder if there is a better way.

Thanks in advance. This package rocks!

[vsivsi]

Hi, there's no "official" way to do this. Although it would probably be more "normal" to write the results of a job using the job.done(result) pattern. The result can then be read as:

Jobs.findOne(job.doc.resolved[0]).result

Another pattern I use is to "prearrange" an _id in another collection (like a file-collection) where the result will be written. Just crate a new mongo ID, and then write it into the data of both jobs, and then the result / data connection is made out of band from the job-collection. Obviously this is really the only way if the data to pass is > 15MB or so.

[ThadeuLuz]

Probably result is the way to go for me because the data is temporary. Thanks

[ThadeuLuz]

Just a quick update: I was having problems getting the descendant task to re-run after it had failed the first time, so I ended up just creating the second task ('descendant') right before the first ('atecedent') finishes. That way I can just add the temp data as usual, on the job.data. Saves me an update and a find.

[vsivsi]

Do you think there's a bug somewhere in retrying failed jobs that were initially delayed waiting for other jobs? Can you elaborate a bit on the problems you saw?

[ThadeuLuz]

Nevermind. Just had to wait for about 5 minutes and the failed job started again. No bugs.

[vsivsi]

Great! Thanks for reporting back.