Gradient test case may fail in GPU single precision mode

[ychfan]

When running the test case in src/caffe/test/test_st_layer_hard.cpp, some cases in GPU single precision mode may fail.

[daerduoCarey]

Yes, I found the same problem when I did my experiments.

I think it’s the precision problem at the point of sub gradient. Just use the double precision mode and you should be fine.

Thank you.

[hugguofeng]

I have the same question, and how did you solve it? When I make test, I see the question.

[daerduoCarey]

I didn't solve it. As i said before, I think that's the inherent precision problem. I chose to use double precision float during my experiments. Thank you.

[seed93]

@daerduoCarey double precision also fails in some cases. When theta is set to identity matrix, unit test fails.

        Dtype *theta = this->blob_theta_->mutable_cpu_data();
        caffe_set(this->blob_theta_->count(), Dtype(0), theta);
        theta[0] += 1;
        theta[4] += 1;
        theta[6] += 1;
        theta[10] += 1;