es6 backends and recline

[starsinmypockets]

Hey folks - Over the last couple weeks I have been working on porting some of the recline data backends to es6!

The projects are here https://www.npmjs.com/~nucivic

Some es6 language features including Promise and fetch mean that these backends can be largely run as standalone modules, without dependencies (an exception being the es2sql library for the cartodb backend).

Given all of this, and some other development initiatives that are underway, including a react-based dashboard which leverages these backends to fetch data and create data visualizations using nvd3, I'm curious if there is any interest in building a next generation of the recline module which could leverage the data backends.

I'm imagining something implemented in es6 that could function as a standalone library (with or without the data backends) and handle common data modeling needs, querying, etc, in the spirit of the recline library, but with a smaller footprint (no backbone, jquery, underscore deps for instance), that could be used in react apps (our primary use case) as well as across other javascript platforms.

[rufuspollock]

@starsinmypockets yes definitely interest in a next generation recline backend.

In general, I think there are 2 things:

A. Would really welcome one or more other persons becoming maintainers of recline - if you were interested (I know nucivic are active here) that would be very welcome.

B. I think we should take a moment to think about overall architecture of recline v2.

On B I have some existing thoughts e.g.:

• Recline should become a "core" with just main models with Views and Backends in their own repos (it might make sense to have a github.com/recline org for this (I already reserved it ...)
• In terms of the core i started some thoughts here: https://github.com/rgrp/dataframe.js (i put it here just because it was very early stage thoughts)
  • I would be quite happy to drop backbone etc - and definitely interested in using react (though core may need very few deps)
• In terms of views I think we just want to document this better and get these in one place and establish a pattern.
  • I have also been thinking about the view "descriptor" language (i.e. the json lang for describing views) - see https://github.com/dataprotocols/dataprotocols/issues/77

/cc @pwalsh @amercader @brew

[pwalsh]

Just a comment on Backbone/React/whatever.

My personal preference is definitely for React, but aside from that, I'm very happy with the approach we've recently taken in OpenSpending, where a little effort in design and implementation has ensured that component logic is not tightly coupled to any framework, and we then have a thin binding for the framework (Angular in that case).

• https://github.com/openspending/babbage.ui/tree/feature/modern/src/components
• https://github.com/openspending/babbage.ui/tree/feature/modern/src/bindings

This pattern enables the possibility of shipping bindings for several js frameworks, and not tying the core lib to any one in particular.

[rufuspollock]

@starsinmypockets any thoughts?

[starsinmypockets]

I haven't had a change to review the code @pwalsh posted but I like the sound and the approach.

We're discussing next steps internally (and have heads down on a data visualization product!)

I am not familiar with the entire recline feature set, but we are using something internally to normalize data from disparate data sources (backends) and performing queries against them (either against the remote data source itself or a against loaded memoryStore). It's largely modeled off of recline. The philosophy is to leverage es6 and to eliminate dependencies, and to remain very framework agnostic:

framework to load and query backends: https://github.com/NuCivic/react-dashboard/tree/master/src/models

backends: https://github.com/NuCivic/csv-es6-data-backend https://github.com/NuCivic/csv-cartodb-data-backend https://github.com/NuCivic/dkan-es6-data-backend

[pwalsh]

Hey @starsinmypockets

Your points on data source backends are well heard here. See my comments on on a related codebase for some recent discussion on this. Looking at your code you are going in a similar direction there.

Side note: I'm a little confused by your terminology around "es6-*" backends. The fact that the libs are written in es6 does not make it an "es6 backend" as I'd understand it: it all gets compiled to es5 anyway in the browser, right? I'm bringing this up not to engage in bikeshedding, but because I was genuinely trying to understand what it means.

In any event, I've been writing quite a lot of ES6 and it is always a pleasure to read other ES6 code - I'd love to collaborate specifically on the "query interface + multiple backends" stuff quite directly, if possible.

[starsinmypockets]

Agreed that the naming is a bit obtuse. I'm open to suggestions!