Possible model improvements / optimisations

[wehlutyk]

None of these are necessary, I'm just listing them to keep track of all the possible small-ish changes that might improve performance or reduce the need for resources:

Normalizations:

• [ ] Batch normalization
• [ ] Self-normalization

Connections:

• [ ] Residual connections

Reduction of necessary parameters and resources:

• [x] Mini-batching
• [ ] Small sampling of neighbourhoods

Noise for training:

• [ ] Dropout
• [ ] Denoising, adapted to Variational Autoencoders

Removing unneeded left-overs:

• [ ] Bayesian compression

Different models:

• [ ] Generative Adversarial Networks
• [ ] Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks

[wehlutyk]

How Powerful are Graph Neural Networks? uses batch-normalisation at all layers.