hallvardnmbu / neurons

Modular neural networks from scratch, in Rust.
https://crates.io/crates/neurons
GNU General Public License v3.0
7 stars 1 forks source link

Combine tensor operations #20

Closed hallvardnmbu closed 2 months ago

hallvardnmbu commented 2 months ago

Look into optimizer steps (and other occurencies), and combine into single operation. Currently "duplicated" looping:

X.mul_scalar_inplace(self.beta1); X.add_inplace(&gradients.mul_scalar(1.0 - self.beta1));