Presentation of results

[msimet]

We need to think about how results will be shown once the tests are run. I think there was consensus that we should output a data file containing the data shown in the plot for people who wish to use other plotting programs, but it would be nice to generate plots automatically (or on demand) as well.

Ideas so far:

• Matplotlib
  • Pro: familiar to most scientific users
  • Con: Slow
• displayQA from the LSST/HSC code
  • Pro: Interactive; plots on demand so it wastes less computer time making unimportant plots
  • Con: Requires ability to run web server (which at least @rmandelb and I won't have on the computer where our data lives)

Or, obviously, both of those. But in either case we'll need to write code that interfaces with those packages.

[alexieleauthaud]

Where are we planning to run Stile? Steve said he could get you an account on Master. If we run this on Master then perhaps we could use some version of displayQA?

[msimet]

Well, there's the issue of where we run it for HSC, but also the issue of where other people in other projects will be running it. The fact that our cluster won't allow a server was meant as an example, not as a specific problem which needs to be worked around.

I think actually my last line (we support both things) is probably the way to go; there's some nice interlinked functionality in DisplayQA, but I can also see some early-level testing where one might want a human to look at every plot that could possibly be produced.

[msimet]

OK, I think Jim and Hironao have convinced me that we should have matplotlib as a dependency. :)

I think we'll still want a plotting command-line switch/driver function kwarg that can be set to matplotlib/displayQA/none, but we can handle some of that down the line.