Tasks with start_state=null not working

[maxrevilo]

I am having problems with the library using the getting started example. The queueWorker doesn't receive more than 2 or 3 tasks, and only one or two are processed. If I don't index _state then it works.

I realised that the problem was on doing orderBy('_state').equalTo(null).on('child_added', ....). For some reason it doesn't work any more, it only brings one or two new childs.

If I set the spec with start_state: 'something' and I create tasks with '_state': 'something' it works like a charm.

Can someone try the example on the readme indexing '_state'? to see if I am the only one. I have other projects in production with firebase-queue and for some reason all of them are working.

[cbraynor]

What version of node / firebase / firebase-queue are you running?

[maxrevilo]

Node: v0.12.7 Firebase: 2.4.0 Firebase Queue: 1.2.1

[cbraynor]

Can you try again - we fixed an issue in the realtime database yesterday that had been causing similar issues, so hopefully it's fixed.

[maxrevilo]

Hi Chris,

Unfortunately problem persists. Maybe you are deploying in staged rollout?

This is the code I am using to test the problem:

var Queue = require('firebase-queue'),
    Firebase = require('firebase');

var ref = new Firebase('https://big-bucket.firebaseio.com/queue-test/');

var tasts = {}
for(var i = 0; i < 100; i++) tasts['task'+i] = {'name': 'task ' + i};

ref
.child("tasks")
.set(tasts, function(error) {
    if(error) throw new Error(error);

    console.log('100 tasks set on firebase');

    var queue = new Queue(ref, {numWorkers: 1}, function(data, progress, resolve, reject) {
        // Read and process task data
        console.log("Working job " + data.name);
        // Do some work
        progress(50);

        // Finish the task asynchronously
        setTimeout(function() {
            console.log("Working done " + data.name);
            resolve();
        }, 1000);
    });

    process.on('uncaughtException', function(err) {
      console.log("Caught exception: ", err);
    });
});

To reproduce the error without Firebase Queue you can replace al the new Queue logic by this

ref
    .child("tasks")
    .orderByChild('_state')
    .equalTo(null)
    .limitToFirst(1)
    .on("child_added", function (snapshot) {

        console.log("New child " + snapshot.key());

        snapshot.ref().child("_state")
        .transaction(function (error) {

            return 'proccessing';

        }, function(error, committed, stateSnapshot) {

            if(error) {
                console.error(error);
            } else if (!committed) {
                console.log('We aborted the transaction (because wilma already exists).');
            } else {
                console.log("State set for Child " + snapshot.key());

                setTimeout(function () {

                    console.log("Removing Child " + snapshot.key());

                    snapshot.ref().transaction(function (task) {
                        if (task != null) return {};
                        else {
                            return task;
                        }
                    }, function (error, commited, snapshot) {
                        if (error) {
                            console.error(error);
                        } else {
                            console.log("Removed child " + snapshot.key());
                        }
                    });

                }, 1000);
            }
        }, false);
    }, function (error) {
        console.error(error);
    });

This is how my package.json looks like:

{
  "name": "queue-test",
  "version": "1.0.0",
  "main": "app.js",
  "dependencies": {
    "firebase": "2.4.0",
    "firebase-queue": "1.2.1"
  }
}

[chevsky]

+1

[cbraynor]

On Friday we released another database fix that was causing issues with losing items at the start of a query (the limitToFirst(1)). Can you verify that it works for you now - when I ran your code it seemed to process all the tasks just fine (with and without a .indexOn)

[maxrevilo]

Sorry for the late response, yes, it's working now, thanks.