Reusable general-purpose query builder

[alanbernstein]

I'd rather put this on the pilosa repo, but I think not all the info here is appropriate for that.

@yuce's PR at https://github.com/pilosa/demo-taxi/pull/3 expands the "Intersect" tab to allow intersections of (unions within a frame). This makes the intersect queries more practically meaningful because they work on ranges of numerical values instead of single values. This is an improvement, but it's also a special-case step in the direction of a general query builder. Our next efforts in front-end work should at least be aware of that goal.

This issue is already ticketed at https://github.com/pilosa/pilosa/issues/240. That gives us a good starting point for building a general-purpose app to demonstrate the power of Pilosa. We will also want to expand on that. With a little planning, we can identify lower-level reusable components and build those as the need arises.

Ideally, we could use the same components for both the webUI and this demo app (or others - maybe a demo section on pilosa.com?). This is blocked by a few things:

• Framework: the webUI was built in startup mode, and with no dependencies. Eventually, this will likely change to use some sort of framework, build process, or other dependencies. We still need to evaluate options here. Ideally, any work we do prior to that should be adaptable to the outcome of that decision. Presumably we will prefer using the same system for both webUI and demo apps. However:
• The webUI framework decision is subject to licensing (and embedding) considerations, which won't necessarily apply to this app or other demo apps.
• The "TopN" tab presents histograms for each frame, which depends on understanding the schema of each frame. To work in the webUI, with an arbitrary index and frame, the app needs some way to retrieve more info about the schema from the server. This is not yet implemented, but there are several proposals about it: https://github.com/pilosa/pilosa/issues/316, https://github.com/pilosa/pilosa/issues/538, and an upcoming proposal from me about reorganizing and extending the content of the /schema and /status endpoints.

[yuce]

I think this is a great note to start the discussion on UI development for Pilosa. In addition to @alanbernstein 's thoughts:

We have a (currently under review) way of adding arbitrary handlers to the Pilosa server at the compile time in the form of handler plugins. I think both the taxi demo and webui (and even docs) are perfect candidates to be packaged as handler plugins. Packaging taxi demo and webui (probably docs too) as plugin bundles has the following advantages:

• The development of those components would be in separate git repos which keeps Pilosa server repo lean (bringing front-end development into Pilosa server repo will cause LOTS of files to be introduced)
• The development of webui/demos can progress independently of Pilosa server
• Users can choose to include in their Pilosa binary whatever they need/want to include (e.g., we can include demo plugins when we are showing off Pilosa; the devs would include docs plugin which is not very useful on a production setup, etc.)
• It's almost trivial to install plugins, e.g., make install-plugin from=github.com/pilosa/demo-taxi)
• There's no performance penalty for using plugins and they are very easy to develop.
• Tight integration with the Pilosa server would be beneficial.