Add more normalisation methods

nedtaylor / athena

A Fortran-based feed-forward neural network library. Whilst this library currently has a focus on 3D convolutional neural networks (CNNs), it can handle most standard hidden layer forms of neural networks, with the plan to integrate more.

MIT License

18 stars 2 forks source link

Add more normalisation methods #4

Open nedtaylor opened 9 months ago

nedtaylor commented 9 months ago

Include more methods of normalisation

[ ] Change name of normalisation procedures
[ ] Change 'norm' argument in renormalise_sum
[ ] robust scaling (uses IQR instead of range from min-max scaling)
[ ] log transformation

nedtaylor commented 9 months ago

Not sure anymore what is actually wanted here. Most likely that we want to convert normalisation into preprocessing layers like Keras: https://keras.io/api/layers/preprocessing_layers/

Resizing Layer
Rescaling Layer
Normalisation Layer
Logarithmic Layer
Discretisation Layer

nedtaylor commented 9 months ago

In the current form, we would need a layer for each dimensionality.

It'd be best if we just converted these ones into using rank 2 data (num_features, batch_size) and then, for get_output and get_gradient, we passed the rank that was assumed. It is meant to be a preprocessing layer, after all, so could just assume rank 2 and have it always feed into input1d_layer_type.

As other layers no longer take in assumed shapes, we should be able to handle this fine with mismatched ranks.