Problem when creating layers with more neurons than the previous layer.

Vector range problem.

Despite the fix you implemented(watched your video too!), the bug still persists. The problem was that the matrix deltaWeights in Backpropagation() did not have the right dimensions to be looped over in:

for(int r = 0; r < tempNewWeights->getNumRows(); r++) { for(int c = 0; c < tempNewWeights->getNumCols(); c++) { double originalValue = this->weightMatrices.at(i - 1)->getValue(r, c); double deltaValue = deltaWeights->getValue(r, c);

            originalValue = this->momentum * originalValue;
            deltaValue    = this->learningRate * deltaValue;

            tempNewWeights->setValue(r, c, (originalValue - deltaValue));
        }

}

(I couldn't get 'insert code' to work, sorry about the plain text)

The bug arises whenever deltaWeights matrix was of smaller dimensions than the tempNewWeights matrix.

The solution was to resize the deltaWeight Matrix to the same size of tempNewWeights before entering the nested loops above.

I've added a function to the Matrix source files that returns a resized array and hence the fixed code loop looks like:

Matrix* resizedDeltaWeights = deltaWeights->resize( tempNewWeights->getNumRows(), tempNewWeights->getNumCols() );

            originalValue = this->momentum * originalValue;
            deltaValue    = this->learningRate * deltaValue;

            tempNewWeights->setValue(r, c, (originalValue - deltaValue));
        }

}

And the implementation of resize is:

Matrix Matrix::resize(unsigned long newRows, unsigned long newCols) { Matrix m = new Matrix(newRows, newCols, false);

unsigned long j = 0;

for(unsigned long i = 0; i < newRows; i++) {

    //if we've reached the boundry rows of the original mat, no left to copy.
    if (i > this->getNumRows() - 1) {
        m->setValue(i, j, 0);
    }else{
        for(unsigned long j = 0;j < newCols - 1; j++) {
            //if we've reached the boundry cols of the original mat, Set values beyond to zero.
            if (j >= this->getNumCols()) {
                m->setValue(i, j, 0);
            }else{
                m->setValue(i, j, this->getValue(i, j));
            }

        }

    }

}

return m;

} Of course, delete resizedDeltaWeights later on. Currently testing this implementation and it works well so far.

ralampay / ann

Problem when creating layers with more neurons than the previous layer. #21