Retrieve job id from within callback

[iclems]

I personally end up using it sometimes in my Firebase workers. It's definitely possible to do achieve the same results differently, so I'd understand if not enough people are interested in this!

[cbraynor]

Its omission is somewhat deliberate - I specifically wanted to abstract the actual task item's location away from task workers to ensure that any updates to it go through a check for whether the current item is still owned by that specific worker - we provide the current state of the task when the process function is called, and allow you to write out the new state at the end by passing it into the resolve() function. The progress() function also allows limited mutation of the current task.

Forces at work beyond the scope of the individual worker can cause it to no longer be responsible for a task it once was. If the location is written to directly it can cause inconsistencies in the task's state.

The client that added the item to the queue will know the task's location, and can always add it to the contents of the task if it's absolutely necessary, but I would caution against this. I think one solution to the issue of listening for completion is to create a second task spec that is chained to the first by specifying its start_state to the finished_state of your first task spec.

If you have a handy example of a specific use-case I'd certainly consider adding the functionality, but my intentions were to reduce the possibility of being able to introduce new race-conditions

[cbraynor]

Feel free to open this back up with a use-case. For debugging issues, the queue itself logs which task is being processed by which worker, and the client creating the job can always put the ID of the task in the task body if no better, application-specific ID can be used or generated

[TedAvery]

I'm pretty new to all of this, so maybe there's a better solution, but I want to get the ID so I can follow up on that specific job later.

It's a very simple search box with no authentication. The client enters a query, the query is pushed to the queue, and the client stores the ID of the job it just pushed. Once my server is finished it's work on the queue, it puts the results in /results/:job_id. My client already knows to listen at that path, based on the ID that was returned when it pushed to the queue.

Sure, I could make my own unique ID first. Or, from the client, I could first push to /results/ and get an ID that I include in the job data itself, which the queue server uses to publish the results to. But that's two queries from the client, when I could do it all in one if the server's callback function could access the unique job ID that was already created :)

[katowulf]

@TedAvery Is there some reason you can't just use the same Job ID as the queued task?

// Using the web JavaScript client
var ref = firebase.database().ref('queue/tasks').push({'foo': 'bar'});

firebase.database().ref('/results/' + ref.key).on('value', function(snap) {
   if( snap.val() !== null ) {
       console.log('Got results', snap.val());
  }
});

Also, as a housekeeping note: Please create your own thread to ask tangentially related questions, rather than replying to resolved threads and diluting the discussion (and the searchability of your questions for others).

[TedAvery]

@katowulf My request is the same as the OP: "Retrieve job id from within callback". Thought it made sense to keep it here, sorry.

As for your example, the queue server callback doesn't have the job ID, because it's not accessible from the callback. So there's no way to know to write to '/results/'+ref.key.

var queue = new Queue(queueRef, function(data, progress, resolve, reject) {
    // Read and process task data
    console.log(data);

    // do work here

    // THE ISSUE: Can't do this. data._id isn't accessible in this callback. Can't get the job ID.
    resultsRef.child(data._id).set({"results" : "here"});

  });

[cbraynor]

See #52 - in short set the sanitize worker option to false and data._id is available. Docs here

[rainabba]

Thanks for the sanitize option. I was in a similar situation and that was a good solution for me. My use-case was moving the task (using a spec and separate queue) to a totally different que (different location in DB), but maintaining a reference for the sake of the clients which are both submitting to a queue (write access) and monitoring another queue (read-only) for related data. Without sanitize, i would have been adding my own UUID and tracking by it, but that would be redundant to the _id (assuming _id is sufficient to avoid collisions inside my own branches of this db).