Investigate errors during training

[lucpaoli]

Two types of error:

• Negative number inside a log
• Loss function becomes nan

Understand what regions of parameter space cause this and how to avoid them. Would we want to bound the parameters or introduce some form of barrier function in the loss?

[lucpaoli]

How can we design our neural net to initialise in the "correct" region of parameter space

[lucpaoli]

Haven't encountered since, interesting discussions wrt bounding the outputs of the NN can be had, though. e.g. tanh to bound between nominal bounds used in cambridge ML_SAFT paper