Added Batch Normalization to Multi_Layer_ANN Model

[vaish-navi-n]

Description

This PR implements Batch Normalization between the hidden layers of the Multi_Layer_ANN model to enhance training stability and speed. The implementation is compatible with both CPU (NumPy) and GPU (CuPy) environments, ensuring versatility across different hardware setups.

Fixes #4

Changes Overview

• Added BatchNormalization class in pydeepflow/batch_normalization.py
• Modified Multi_Layer_ANN class in model.py to incorporate batch normalization
• Updated forward propagation and backpropagation methods to include batch normalization
• Implemented running mean and variance for inference
• Added compatibility layer for NumPy and CuPy operations

Key Implementation Details

1. BatchNormalization class:

   • Initializes parameters (gamma, beta) and running statistics
   • Implements forward pass with training and inference modes
   • Implements backward pass for gradient computation

2. Multi_Layer_ANN modifications:

   • Added use_batch_norm parameter to include batch normalization in the model initialization
   • Modified layer structure to incorporate batch normalization layers
   • Updated forward and backward propagation to handle batch normalization

Testing

• Added unit tests for BatchNormalization class

[ravin-d-27]

Hi @vaish-navi-n ! Thank you for your contribution !