search

tsoding / nn.h

Simple stb-style header-only library for Neural Networks
MIT License
342 stars 35 forks source link

BackProp: Divide by (n-j) instead of (n) #4

Open agora-97 opened 1 year ago

agora-97 commented 1 year ago

I think in nn.h L351 and L356, correct thing to do is divide by (n-j) instead of just (n). Adder of 2 BITS using 10K iterations improves from aprox .0006 to .0005 I'm not into maths neither AI, so probably I'm wrong xd

Another thing, should we drivide by n-j activations like so?:

    for (size_t j = 0; j < g.bs[i].rows; ++j) {
        for (size_t k = 0; k < g.bs[i].cols; ++k) {
            MAT_AT(g.as[i], j, k) /= (n-j);
        }
    }