Auto refresh metric plot

[jrao1]

Does this feature exists? Right now if I want to refresh the plot I had to click Metric plots menu item again, then select the checkboxes again.

I can see the sacredboard calls /api/run/x periodically, so it should be getting the latest data?

[chovanecm]

Hello, unfortunately, not. The call is currently only used for updating the captured standard output. But it sounds like a good idea.

[talesa]

@chovanecm could you give some directions what to look at to implement this?

[chovanecm]

@talesa Thank you being interested in having a look at it!

I will first try to explain you the design. Maybe you don't get everything from my description in the beginning, so feel free to tell me your thoughts after reading it.

Basically, Sacred stores metrics as an array of x and y values directly to the database (which has, by the way, certain limitations in the record size, by default 16 MB or so). Sacredboard exposes those metrics through API at http://localhost:5000/api/run/33/metric/ID_OF_THE_METRIC

The front end is mostly made of KnockoutJS components that define interface for the outer world and encapsulate logic. They usually consist of an HTML template and a JavaScript file that defines a view model that uses a lot of Observables or Observable arrays. Elements in the page are bound to the observables so that they should update automatically if something in the view model changes.

For instance, the top component of Metrics that you see in the detail view is called metrics-viewer and takes a sacred run object as a parameter (It is a part of the JSON returned by http://localhost:5000/api/run/RUN_ID).

Inserting that code triggers the constructor of the metrics-viewer view-model. Then it iterates over the available metrics in the run and for each of them, a proxy object that lazily loads their values via HTTP is created. You can notice that the created objects are stored in an observable array

this.metrics = ko.observableArray();

The <metrics-viewer ...> component itself inserts its template to the page and passes the newly created metrics as a parameter named availableMetrics to the metricsPlot component that is shielded from any loading logic and can be use completely separately as in this test page, which you can open (when sacredboard is running) at http://localhost:5000/static/scripts/plot/test.html It is responsible for rendering the user controls (axis type, list of metrics) and holds lists of available and currently selected metrics. It also converts the domain-specific metric to a generic series object that can be finally consumed by the plot-chart component. Also note that the conversion happens automatically when a metric is added to the observable array of currently selected metrics.

What format of data the plot component expects can be again seen in its corresponding test.html file. The current implementation assumes that a series (metric) can be added or removed, which causes the plot to redraw. The Plot interface tries to be rather generic to allow for different rendering implementations. Currently, Plot.ly is implemented.

But there is currently no handling of changing the x and y values of a series itself. So I think there are two challenges:

• implementing reaction to the underlying x and y changes (no matter where the data come from)
• designing and implementing a refresh mechanism that periodically (or on request?) reloads data from the back end (as it is done only once using the proxy object now) and lets the plot know that the data has changed. Preferably the other components in the "pipe" (metricsplot -> plot-chart) should detect the change using observation of the underlying data rather than having some separate public callback method.

Perhaps the plot-chart component can observe the values of x and y array changes and then redraw the series (simplest but "stupid" solution: remove and add the series again with new values).

I think you'd need to have a look at the KnockoutJS framework. It is simpler than e.g. Angular, but it does not have such a large community (though it is not bad, I should have chosen something that more people know)

By the way, the components are dynamically loaded using RequireJS so that all the javascript includes don't have to be specified on the main page.

[chovanecm]

I have also started writing a development guide to make sacredboard running from source locally: https://github.com/chovanecm/sacredboard/wiki/Development-Guide

[danielcrane]

I'd also appreciate this kind of feature, or at least a 'refresh' button on top of the plot (that doesn't make me tick the check boxes again).

[Leinadj]

Yes, this feature would be awesome. Refreshing everything would be ok, in my opinion, but would be great if the metrics plot is not collapsed and keeps showing

[talesa]

Agrhh, @chovanecm I'm sorry for not following up on this yet, I'm short on time now, but I might come back to this eventually