Enable ingesting numerical attributes with continuous values.
The aim has been to add a continuous numerical attribute to the diagnosis example which adds no additional information. In this case, the model should be able to stably achieve the same performance as without this attribute. Empirically, this has been achieved, perhaps taking longer to converge (about 500 training iterations minimum, compared to a minimum of 250 iterations prior).
Closes #99
What are the changes implemented in this PR?
Introduce a continuous numerical attribute into the diagnosis example (severity)
Create a ContinuousAttribute model, consisting of an MLP followed by layer normalisation
For debugging and monitoring purposes, add histograms at strategic points in the model, plus the relevant code to execute and store these summaries
Normalisation of common (type) embeddings for improved stability, such that attribute and type embeddings have similar magnitudes
Gradient clipping for a very significant improvement in model convergence and stability
Dropout for continuous attribute MLP to combat overfitting
What is the goal of this PR?
Enable ingesting numerical attributes with continuous values.
The aim has been to add a continuous numerical attribute to the diagnosis example which adds no additional information. In this case, the model should be able to stably achieve the same performance as without this attribute. Empirically, this has been achieved, perhaps taking longer to converge (about 500 training iterations minimum, compared to a minimum of 250 iterations prior).
Closes #99
What are the changes implemented in this PR?
severity)ContinuousAttributemodel, consisting of an MLP followed by layer normalisation