Batch Performance

[kinhosz]

Improvements for overall network performance

• [x] Backpropagation in Batch (currently use batch size = 1)
• [ ] Batch normalization:

  $$N(x) = \frac{(x - \mu)}{\sigma} \cdot \alpha + \beta$$

• [ ] Leaky ReLU:

  $$f(x) = \begin{cases} x, & \text{if } x>= 0 \ k \cdot x, & \text{otherwise} \end{cases} $$

These changes should only modify the version patch.

[kinhosz]

14

Para resolver Backpropagation in Batch