[feature request] Graphing facilities for consecutive experiments

[pratiksha]

When using CodaLab as a research notebook, a useful feature would be some basic graphing facilities for consecutive experiments, in which we might want to visualize the effect of varying one parameter on some output. For example, a % display graph x y directive, which would create a scatterplot of x vs. y in the following set of run bundles in the way that % display table displays the results in a table.

[percyliang]

Refinement of spec:

% display graph /output.tsv x=0 y=3 “xlabel=# iterations” “ylabel=test error” header=false
{bundle1}
{bundle2}
...

For each bundle, we would read the TSV file, read the relevant columns, and plot it. This way, we have one curve plotted for each bundle. The various options at the end allow us to customize the plot. The plot should be done in D3, so that we can have more interactivity with the plot.

Main use case: plot the training curves of multiple runs with different hyperparameters.

[percyliang]

After talking to some people, it seems like people will want to have more flexible graphing capabilities, so we should just expose the full functionality of D3 (but still have a good default), so that people can render the output of bundles however they want.

% display graph /output.tsv js="""
var div = null; // The thing that want to display.
for (var i = 0; i < data.length; i++) {  // data is an array of JSONs
   // do arbitrary javascript
}
return div;
"""

Note that we need multiline fields for this.

[pratiksha]

I think the refinement is slightly different from the original request (having ways to manipulate and display the aggregate output of consecutive bundles). I'm not sure if you meant for this functionality to include that, but wanted to check that the two aren't getting confused in this thread.

[percyliang]

Handled by #1442