Abstraction for views waiting on data

[theq629]

The new selections system (#183) has helped to remove some of the callback spaghetti we have tend to have in the view code. I'd like to try to identify and remove other patterns that could be factored out. A remaining obvious spaghetti pattern is waiting for data and then drawing the view.

Current situation

Right now we tend to have something like

var dataFromGlobalQuery = null,
    dataFromLocalQuery = null;
function draw() {
    if (dataFromGlobalQuery != null && dataFromLocalQuery != null) {
        // hide loading indicator, draw view
    } else {
        // show loading indicator, clear view
    }
}
globalQuery.onResult(function (result) {
    if (result.data.hasOwnProperty('error')) {
        dataFromGlobalQuery = null;
        // show error
        draw();
    } else {
        dataFromGlobalQuery = result.data;
        draw();
    }
});
localQuery.onResult(function (result) {
    if (result.data.hasOwnProperty('error')) {
        dataFromLocalQuery = null;
        // show error
        draw();
    } else {
        dataFromLocalQuery = result.data;
        draw();
    }
});

This isn't so bad by itself, but gets pretty messy when combined with UI things that also need shared state that goes to draw().

Basic improved system

My current idea is to add an interface for sources of data, which supports callbacks on data changes including unavailable (still loading) and error states. Query result watchers can implement this interface. We also have a utility which watches several data sources and supports the same callbacks, with the data available state only being triggered when all input sources have available data. Then it's easy to extract out the common pattern of having a view that gets drawn when all data is available, with loading and error indications when it's not. This leads to something like

var consumer = new DataConsumer({
        global: globalQueryResultWatcher,
        local: localQueryResultWatcher
    });
setupDataView(outerElt, consumer, function (data) {
    // draw the view
});

Here DataConsumer is the data-combining utility, and setupDataView handles the loading and error indications and clearing the view, and calls the callback when it can draw the view. The callback gets data with the same properties (global, local) given to DataConsumer for the different input parts. It's also possible to watch consumer directly for data changes in unusual cases (in fact I'm not sure if that's needed; maybe we could merge DataConsumer and setupDataView).

Handling UI components

We can also have a simple wrapper so that UI components important to drawing the view (eg the facet selection in storyline) can easily act as data sources. This means that setupDataView needs to support an additional state for when the user needs to change something in the UI. This state should display a helpful message instead of the loading indicator. I haven't thought through all the details of this, but I think that setupDataView just needs to take an extra callback which gets the current ready/waiting state of each input and either returns a custom message string or a null indicating that only the standard loading message should be used. Alternatively DataConsumer could flag or separate those inputs and output a separate state when waiting on them.

[theq629]

Done in the queriesrefactoring branch, because it seemed easier than refactoring without updating this part. Data sources emit the following events:

invalidated: any previous data becomes invalid error: there was an error in getting data result: new data is available

Paginated data sources also emit 'done' when all pages have been seen.

If we ever added streaming data, it could use the same interface as paginated data sources.

The above example becomes:

var data = new DataSource.Merged({
        global: globalQuery,
        local: localQuery
    });
data.on('result', function (results) {
    // draw the view
});

I've also added adapters for SingleValueSelections so that UI elements can plug into the same merged data source. For this you wrap the value of a UI element in a SingleValueSelection and then pass in an adapter to data source merging like above, and the view updates automatically on any changes.

[theq629]

I'm not totally happy with this. It's definitely an improvement for query results, but the part for UI elements doesn't fix all the spaghetti code and it's getting into a custom system that that may be hard to understand.

However, the UI part ensures consistency (eg fixes some loading indicator status bugs), and it means we could easily save and restore preferences (and maybe automatically log all changes). So overall probably an improvement.

The next step for improving view code would be to look into existing solutions for this sort of thing (possibly FRP libraries) and either use one or use them as guidelines.