Royal Rumble of Plotting Tools

[astrobokonon]

Intro/Scope

This is an attempt to survey what's possible with each potential plotting package. Digging into bokeh a bit more (as part of my work furthering issue #4) I'm wondering if bokeh is the tree we should be barking up and whether there's a better option out there. This issue is an attempt to document some of that searching process.

Since the vast majority of things are time series data such as temperature, humidity, dew point, etc., I've focused on what I feel is most important.

Ranking considerations

• Ease of setup
  • If it's great, but takes two weeks for someone new to make a plot of their own, or if it has a fiddly requirement set, then it's really not great after all
• Responsiveness
  • If it's interactive but laggy and frustrating to watch, it's not great. It should be objectively and obviously "snappy" and quick to update
• Value added controls
  • Selecting a group/range of points and median-ing them would be nifty
• A more fuzzy/subjective style consideration
  • It shouldn't look like it's from 1995, we're aiming for end users/observers for some aspects of this so it should look new, slick, and "good" and should be able to be updated easily.
    • bokeh themes are a great example of what I'm looking for, a simple configuration file that lives outside of the actual code.

Open Questions for All Packages

• If the user changes the time range in one plot, should the time range in the other plots change to match?

  • Pros:
    • Unambiguous cross correlation of information
  • Cons:
    • Maybe you just want to see one thing in particular and this would be annoying
    • Question of what happens on the next timed update interval to grab new/fresh data

• ?

Tools Compared

• bokeh
• plotly
• mpld3 (really just matplotlib with extra flavorings)
• embedding other tools' graphs
  • Not super enthused about it, but Grafana panels can be embedded via an iframe or an API call

[astrobokonon]

bokeh

Pros:

• Integrates easily into django/web friendly by design so no high jumps to get going
• Seems to be snappy and responsive at least locally
• Interactivity is good and well done
  • e.g. supports mouse zoom in a single dimension, and this is configurable even in the YAML theme file
  • webgl backend support so large draws should be way quicker if the users' browser supports it
• Easily theme-able, a simple YAML file can set the defaults for any property of any class
• Popular
• With a good theme, it can look great

Cons:

• It is somehow possible to get a plot like the one above that looks right but is just completely screwed up in some subtle way
  • The label on the tool tips don't make any sense compared to either axis because of the specific mode and specific choice I made for the tool tip. It's all poorly documented with bad examples.
• Documentation examples can be garbage because they leave a lot assumed
  • Highly modular, so it's an import extravaganza to actually get something running and they don't always actually tell you where to import stuff from
• The hover/tool tips are at best per series, and if the series overlap then the tool tips overlap
  • Makes them almost useless for some things, so the statistics are probably best broken out in a table elsewhere
• For a static output, the data is base64 encoded in the JS that's output so it will likely be slow for larger queries
• Might be some need for a bokeh server running locally to generate some plots/plot types? Confusing server/client roles in the documentation as well but it would probably become clearer with experience/practice.

[dyerlytle]

For "Active Plots" on a web page, the bokeh server seems to be necessary. It may be that some other plotting package for web would have a simpler setup, we don't know that yet. Bokeh seems to be widely used, well supported, and the code is Python. I suspect some of the "Cons" above can be overcome as we learn more about the package. The tooltips seem to be fairly configurable but I admit I've only played with them a little.

[dyerlytle]

Also, I agree that the resulting web page/plots should look good. The main thing is to make the information the user needs as clear as possible and easy to see at a glance. To this end, perhaps each different kind of plot should be considered independently to determine what sort of display is best for the information in that plot. On the other hand, if the plots all look substantially different, the overall effect will be a mish-mash and look terrible.

[astrobokonon]

For me, the following GIF shows my ideal for a non-frustrating user interaction with a plot of many different types of information:

The tooltip follows the mouse cursor position but displays the relevant data. Most importantly, the tooltip doesn't block the data you're trying to look at!

More Bokeh Tests

If you make a line plot then the tooltip ALWAYS follows the line. This is great for a time series that is simple, like every example available on the internet. It works both faster and nicer than plotly's equivalent which is also good.

Once you put your real data in, it's all downhill. The "T2" parameter on the ARC/LOIS instruments is a good example of this - it has a short period sinusoidal variation since it's ultimately tied to the facility chiller system which has a refrigeration cycle of its own. This GIF was captured at 30 Hz; it's even worse at the full monitor refresh of 60 Hz.

Switching to a scatter plot instead seemed like a good idea, since the hover/tooltips for those is more flexible. But it turns out that if you have many points near your cursor (because there was a quick change, or your time range compresses things) it just shows you ALL the points that are within some unconfigurable tolerance of the mouse position. It's also not possible to do a connected scatter plot natively - you have to draw the line, then draw the data markers separately. I'm not a fan of doing 2x draws for every single quantity since Bokeh would then embed the data 2x for every series in the resulting JS/HTML.

Both of these are old issues with Bokeh (the original issue dates back to 2014), and not much has really been done since then. My last hope for Bokeh lies in the CustomJSHover method, but if that doesn't improve things then I'm inclined to ditch Bokeh for now and start investigating plotting in JS directly.

[astrobokonon]

This took a large number of obscenities to get sorted out, but it's now complete and I understand why and I like it and I can recreate it for other plots.

Basically, I created a list of patches whose x coordinates were the coordinates of adjacent datapoints with y coordinates that span the initial Y range (for CCD Temp, which is "y1"). I then added those to the main Bokeh ColumnDataSource, plotted them, and set the HoverTool to ONLY act on that set of patches. It's important to put everything in the same ColumnDataSource since the HoverTool can only get values within a single one of those; you can't reference a separate ColumnDataSource.

That makes the tooltips valid, highlight only the points actually in the patch, and it's not that tremendous of a pain to do. I'm happy with this for now, so I'm content to close this issue and finally move on with my life.

[astrobokonon]

Working as of e525327c0d002c1b63ec2fdecf61a08ba51e2482