View/sort/filter/graph benchmarking results.

[jaffee]

pi spawn can store its results as json documents in S3. Currently they all just get dumped into one bucket without much structure.

An interface to benchmark results is needed - this is potentially an infinite rabbit hole, so let's define a sparse MVP:

• Central repository for spawn configs with version tracking.
• Ability to grab a certain set of spawn results - i.e. all results generated from a given spawn config file is probably the most important.
• Given a set of spawn results, have some kind of comparison view which allows one to see how the results have changed sorted by date or Pilosa version.
• I hesitate to include graphing in the MVP, but maybe @alanbernstein can weigh in. As with everything, I'd prefer to keep this in Go, but that may not be realistic.

This will probably take the form of a web app backed by a lightweight SQL database. I imagine we'll want to make it available at perf.pilosa.com or similar. I'll defer to @codysoyland's expertise for deployment and operational considerations.

[alanbernstein]

Some experimentation with graphing would be appropriate. As long as we keep things cleanly separated, I think we can do a small set of static generated png graphs in the MVP. This is easy with, for example, https://github.com/gonum/plot.

After MVP it might be nice to handle some of the sorting/filtering in a light UI, in which case the best graphing choice might be tightly coupled to that.

[alanbernstein]

Other plotting options:

• https://github.com/cznic/plot (gnuplot wrapper)
• https://github.com/plotly/golang-api (seems to only support the online service mode)

[jaffee]

we've got codespeed, and we've put some stuff into data.world and gotten plots out of that, and Alan has some scripts for building plots from the data.world stuff. Going to icebox this for now.

[alanbernstein]

See https://github.com/pilosa/vis for some of those scripts and plots