Plotting functionality for monitors

IDSIA / brainstorm

Fast, flexible and fun neural networks.

Other

1.3k stars 152 forks source link

Plotting functionality for monitors #44

Closed sjoerdvansteenkiste closed 8 years ago

sjoerdvansteenkiste commented 9 years ago

Would it make sense to provide basic plotting functionality to monitors that output per-epoch results (i.e. the accuracy monitor)?

In one glance you would be able to obtain information about (speed of) convergence and performance.
Allows easier sharing of results from one researcher to another.
Would require matplotlib to be in place if the user chooses to make use of this functionality.

Qwlouse commented 9 years ago

Yes absolutely! This would be great as a Hook, that can be told which logger entry(entries) to plot.

flukeskywalker commented 9 years ago

@sjoerdvansteenkiste That's a desirable feature indeed. I also suggest that you look at https://plot.ly/python/ which makes nice plots which can be viewed and updated live in the browser.

Qwlouse commented 9 years ago

I guess we'd need different Hooks for different plotting libraries. I'm not a fan of plot.ly because its proprietary. bokeh would be my choice I guess. But matplotlib is also a reasonable choice.

untom commented 9 years ago

Personally, for mainly this reason (different preferences by different users). I'd leave the plotting to the user. As long as the statistics are easily accessible from the monitors, a quick plot can be generated in a line of code or two.

flukeskywalker commented 9 years ago

I agree with @Qwlouse. Bokeh is preferable. @untom: We can still ship a couple of helper hooks for plotting so that someone new can see how to write their own plotting hook. It's also nice to have something that works out of the box.

sjoerdvansteenkiste commented 9 years ago

Basic accuracy monitoring is now provided in the VisualizeAccuracy Hook. Bokeh Server is required for running. Will continue working on this at a later moment

flukeskywalker commented 9 years ago

@sjoerdvansteenkiste, I renamed it to BokehVisualizer since it can be used to visualize any log. The docstring says that "By default the output saved as a .html file, however a display can be enabled." but it's not clear how. Is there no live display by default? If so, how to enable it?

sjoerdvansteenkiste commented 9 years ago

@flukeskywalker Sorry didn't see your comment earlier.

Will update the doc string later this week to ensure it specifies the right behaviour. Furthermore dynamic plotting is by default enabled (given a bokeh server is launched). And the resulting visualisation will always be written to an .html file at the specified location.

All in all to close this issue all I need to do is some textual updates and perhaps some catching of errors/warnings launched by bokeh. Will work on this likely tomorrow.

sjoerdvansteenkiste commented 8 years ago

Finished Bokeh Visualisation Hook for release.

Warnings fired by Bokeh when a server instance is not running are now caught and a StopIteration will be raised
Updated description strings and added support for when a single string log is supplied
The Bokeh Visualisation Hook now by default streams the log-data to your browser while the network is being trained. The .html visualisation will only be saved on your directory if an output file is supplied.

Closing this Issue for now. Will re-open if something pops up.