search

contribsys / faktory

Language-agnostic persistent background job server
https://contribsys.com/faktory/
Other
5.73k stars 227 forks source link

Can on-complete Job know its own Batch Id? #361

Closed pbrisbin closed 3 years ago

pbrisbin commented 3 years ago

I have an interesting BATCH use case and I'm just wondering if there's a way to accomplish it...

We want to use BATCH to wrap a bunch of Jobs and notify Dead Man's Snitch (DMS) at the end.

When you notify DMS, you can:

Faktory gives us:

I'm not sure how to best map the check-in's I want to perform in DMS to these, since there's no failure Job. Do I check-in neutral from complete and success from success? This is probably what I'll try, but I'm not optimistic about how things will appear in the DMS UI.

This got me thinking, if the complete Job knew its Batch Id, it could use BATCH STATUS to find out what it needs to know and check-in appropriately with DMS all from the one callback. Is this possible?

mperham commented 3 years ago

Faktory does not have any way to push implicit state into a job but the batch callback can take any set of arguments you wish, including the BID.

b = Faktory::Batch.new
b.success = {"jobtype":"MySuccessCallback","args":[b.bid],"queue":"critical"}
...

Typically I would have job errors going to my email so I can fix any issues, DMS not necessary. If you don't have something like that, I would suggest using complete and sending a notification if the batch status shows that there are failures remaining in the batch.

pbrisbin commented 3 years ago

Oh, that approach would certainly work, but can you translate that ruby into the line-protocol?

b = Faktory::Batch.new
# bid = BATCH NEW {"description":"An optional description for the Web UI", "success":{ what's going here? } }

b.success = {"jobtype":"MySuccessCallback","args":[b.bid],"queue":"critical"}
# What command is this then?
mperham commented 3 years ago

Ah right, I was confused. Wiki updated.

https://github.com/contribsys/faktory/wiki/Ent-Batches#callbacks

All callback jobs get special attributes in the custom hash which provide context. Those attributes are not passed to the callback job as arguments but you can set them as thread-local variables (implicit state) or dynamically set an accessor on the job.

pbrisbin commented 3 years ago

Callback jobs automatically have custom _bid ... set with their associated Batch ID ...

That's exactly what I was wondering, thanks!