fix dropout backward error, some inconsistencies

[kikaxa]

see https://github.com/hughperkins/DeepCL/issues/100 the key fix is in DropoutLayer::backward() for maskWrapper - it gets off the device somehow after forward()?

[hughperkins]

Cool. Some kind of unit-test, so I can compare before/afterwards?

[hughperkins]

Unit-tests are written using google test by the way, eg https://github.com/hughperkins/DeepCL/blob/master/test/testdropoutbackward.cpp#L20

TEST( testdropoutbackward, basic ) {
    int batchSize = 1;
    int numPlanes = 1;
    int imageSize = 3;
    EasyCL *cl = DeepCLGtestGlobals_createEasyCL();
    DropoutBackward *dropoutBackprop = DropoutBackward::instanceForTest( cl, numPlanes, imageSize, 0.6f );
    uchar mask[] = {
        1,1,0,
        0,1,1,
        1,0,1
    };
    float errors[] = {
        3, 5,-2.7f,
        2, -9, 2.1f,
        0, -1.1f, 3.5f
    };
    int inputTotalSize = dropoutBackprop->getInputNumElements( batchSize );
    EXPECT_EQ( batchSize * imageSize * imageSize, inputTotalSize );
    float *errorsForUpstream = new float[ inputTotalSize ];

    dropoutBackprop->backward( batchSize, mask, errors, errorsForUpstream );

    EXPECT_FLOAT_NEAR( 3, errorsForUpstream[0] );
    EXPECT_FLOAT_NEAR( 5, errorsForUpstream[1] );
    EXPECT_FLOAT_NEAR( 0, errorsForUpstream[2] );

    EXPECT_FLOAT_NEAR( 0, errorsForUpstream[3] );
    EXPECT_FLOAT_NEAR( -9, errorsForUpstream[4] );
    EXPECT_FLOAT_NEAR( 2.1f, errorsForUpstream[5] );

    EXPECT_FLOAT_NEAR( 0, errorsForUpstream[6] );
    EXPECT_FLOAT_NEAR( 0, errorsForUpstream[7] );
    EXPECT_FLOAT_NEAR( 3.5f, errorsForUpstream[8] );
//    for( int i = 0; i < 16; i++ ) {
//        EXPECT_FLOAT_NEAR( expectedErrorsForUpstream[i], errorsForUpstream[i] );
//    }

    delete dropoutBackprop;
    delete[] errorsForUpstream;
    delete cl;
}

[kikaxa]

hmm. i'll see what i can do tomorrow.

existing unit tests always pass for me, on both cpu and gpu. test network fails on gpu only.

[hughperkins]

Cool. Seems convincing. Thanks! :-)