Simplify App Development

[gka]

This is a proposal for simplifying the development of PyBossa apps. This is closely related to the recent discovery that it's a good idea to pre-load tasks in order to speed up PyBossa apps. What this showed is that more and more logic is needed to write smart apps, and I recommend that we should take this burden away from the users.

For one, we really need to version the app "framework", to ensure that past apps won't break as we change it. So apps should get a new field which defines how the presenter should be interpreted.

Now, let's have a look at the proposal:

Essentially there are only two hooks a user need to plug into when writing a new PyBossa app: taskLoaded and presentTask.

taskLoaded

In taskLoaded, the app gets a task object that has just been loaded (or pre-loaded) along with a deferred. Here you can load additional data from other APIs or fetch some images, and as you're ready you resolve the deferred.

In the most simplest case, no additional data is needed and the deferred can be resolved immediately. This should be the fallback if the user does not define taskLoaded himself.

pybossa.taskLoaded(function(task, deferred) {
    deferred.resolve(task);
});

Here's a more sophisticated example, in which an API is called and the response is put into task.info dictionary.

pybossa.taskLoaded(function(task, deferred) {
    // load some data from an API
    $.ajax({
        url: 'http//api.example.org/entity/' + task.info.entity_id,
        dataType: 'jsonp'
    }).done(function(response) {
        $.extend(task.info, response);
        deferred.resolve(task);
    });
});

In the flickrperson app, taskLoaded should take care about pre-fetching the next image from Flickr. This can be done by inserting an invisible image element to the DOM, maybe moved outside the viewport or with opacity set to 0. The image element itself can be stored in the task.info blob which is passed to the presenter.

presentTask

Almost the same architecture can be used for presentTask hook. You you get a task object (which eventually has been extended by taskLoaded) and a deferred.

In this function you update your UI so it presents the task properly. You set your event handlers and eventually resolve the deferred as the user has come up with a solution.

pybossa.presentTask(function(task, deferred) {
   // present task to user
   $('.my-ui').html(task.info.title);
   //  set-up event handlers
   $('.my-submit-btn').off('click').on('click', function() {
        // pull the answer from UI
        var answer = $('.my-answer-field').val();
        deferred.resolve(answer);
   });
});

the rest should be magic

The entire rest of the magic should be handled by pybossa.js, without any line of code in user apps. pybossa.js would ensure that tasks are pre-loaded, that the url hash for the task is updated, the answers are submitted etc. If the presentTask-deferred is resolved without passing an answer we assume the user has skipped this task.

Of course, we could introduce hooks for special events, e.g. all tasks have been completed, but I argue that there should be a fallback, so implementing this is optional.

What do you think about this? Do you need more examples?

Open for discussion.

[gka]

Here's how the flickrperson app would roughly look like using the proposed app framework: https://gist.github.com/4280721

And here you can see the new flickrperson app up and running: http://crowdcrafting.org/app/flickrperson2/newtask (src: https://github.com/gka/pybossa-app-flickrperson2)

[teleyinex]

Hi,

I like this idea a lot! We should improve pybossa.js by all means providing this approach for all the new apps :+1:

Questions: In the beginning PyBossa only supported one page for doing tasks, so using this approach is going back to that days, hehe. I changed this behavior basically because we wanted to provide forums or discussion threads per task. This is very common in platforms like GalaxyZoo and it has been a really important feature for new discoveries, and good discussions. We also know that forums are good from the volunteer computing communities as there are a lot of comments and feedback between the users and the owners of the application.

While this feature will not be used by every application developer I would like to, at least, provide a simple solution for this. Since the beginning I've been thinking about two different approaches:

• Develop our own PyBossa forums, in the core system (this will basically make things simple at the cost of re-inventing the wheel)
• Use Disqus, the only problem, as far as I know, is that we have to provide unique URLs for each task, so the pre-loading will be completely lost :-( (I tried already with the location hash, but for Disqus it was the same URL).

Therefore, do you know any solution where we could use both approaches? I've been looking for a flask-forum extension, but no luck so far (nothing really useful). Therefore, I strongly support your proposal, and I would like to mix it with the Disqus thing :-)

[teleyinex]

Another problem of using Disqus is that users that have signed in via Twitter or whatever, they have to signin again just to post a comment, so maybe it is really the time to create a dead simple forum for PyBossa (see this issue). Comments?

[rufuspollock]

@teleyinex I think we do want to update the url (among other things it means we can link back to a task) but I'm not sure this is contradicted strictly by the above - the JS could take care of updating the disqus forum and we can use pushstate to have nice urls.

[teleyinex]

@rgrp good point! :+1: I think it is worthy to link here the docs of pushstate. I'll have a look at this and see if it fits with disqus :-)

[teleyinex]

Hi again, pushstate works like a charm :-) plus disqus uses it correctly, so we do not have to actually change so much our own code, and enabling the pre-fetching will be really simple, so what do you think? Do we support in pybossa.js what @gka has suggested?

[teleyinex]

Another interesting tip: without using pre-loading and only pushstate the speed of loading new tasks for FlickrPerson finder is almost similar (with smaller images, obviously) to the pre-loading one :-) I'm amazed!! I've updated flickr person finder with only pushstate so you can actually compare the difference. It is amazing how fast it is :-)

[teleyinex]

@gka @rgrp I think that we can include the snippets of code that @gka has developed and used in flickrperson2 so everyone can start using this approach. I don't think that this will break old apps, but it will be worthy to check it carefully. @gka comments on this?

[rufuspollock]

Can we get this merged :-) I'd like to start using this :-)

[teleyinex]

@rgrp sure! Let me finish first the codemirror code for PyBossa and then I'll work on this item.

[rufuspollock]

Where did codemirror come into things? BTW I've just done some codemirror work as part of Data Explorer: https://github.com/okfn/dataexplorer/blob/master/src/views/project.js#L155 (this also includes the ability to run the code in a sandbox ...)

Want to see it in action just boot a project on Data Explorer and open the script editor ... (e.g. visit http://explorer.datahub.io/?url=http://resources.opendatalabs.org/datasets/standard-and-poors-500-shiller/data/data.csv&backend=csv)

[teleyinex]

@rgrp sorry, I explained myself badly :-) I'm working in the PyBossa issue 142 for allowing users to create apps and write the task presenter directly in PyBossa, not using the API. This will basically allow users to create an app with these steps:

• Sign in
• Click in the Create link
• Add some info about the app
• Upload some tasks with the CSV importer (using GDocs)
• Create the presenter in the web

I'll have a look to your code!

[teleyinex]

@rgrp @gka here you have the first version of the new pybossa.js solution proposed by @gka. As you will see, I've modified a bit the initial example by @gka as I prefer to use the $.ajax method instead of $.getJSON as it keeps, IMHO, everything more clear (instead of really long URIs).

The only missing part is the unit tests. The new proposal uses several deferred objects, so we will need to do some stubbing, or something similar, so any help is more than welcome, as I'm not sure yet how to implement that unit tests. If you know how I should do it, let me know it.

Please, review the code also!

[teleyinex]

Hi! Good news, I've found the way to do the testing :-), so do not worry too much.

[rufuspollock]

@teleyinex use qunit + sinon.js :-)

[teleyinex]

@rgrp it is what I've been using since the creation of PyBossa.JS :-)