Plot parameters during training

[flipdazed]

Aim Be able to visualise the parameters as training occurs - identify if gradients are blowing up

Outcome Plot cost vs. minibatch similar to https://deeplearning4j.org/visualization Plot magnitude of gradients as they are multi-dimensional

[flipdazed]

Need to:

• take magnitude
• take mean

[flipdazed]

Created class utils.visualise.Visualise_Runtime to hold data at run time and plot

Store data in python lists and save in HDF5 format with hickle or png with Pillow

• weight images saved as .png
• cost saved as .hkl
• arbitrary parameters saved as .hkl

[flipdazed]

• The two list structures: visualise_weights and visualise_params should be dict now that 'name' is simplified
• Remove plot_dir and data_dir as arguments to train()
• Remove the overhead of visualisation out of train() and into Visualise_Runtime()

[flipdazed]

Had to stop using hickle as it was causing errors for large arrays

[flipdazed]

Test with test.test_mlp and gnuplot