search

NVIDIA / caffe

Caffe: a fast open framework for deep learning.
http://caffe.berkeleyvision.org/
Other
672 stars 263 forks source link

can't reproduce testing accuracy in the log file #516

Closed lixincn2015 closed 6 years ago

lixincn2015 commented 6 years ago

Hi, I've tested the trained caffemodel, and the accuracy is consistently ~0.5% lower than that claimed in the log file. What could possibly be wrong?

drnikolaev commented 6 years ago

Hi @lixincn2015 let's start with logs, please upload them here

1duo commented 6 years ago

Hi @drnikolaev, I observed same issue here. Basically during train/val (caffe train -solver ...), when it does validation, output includes both validation accuracy and at the meantime, a snapshot, using SnapshotWithScores(), with accuracy score in filename (https://github.com/aNVIDIA/caffe/blob/caffe-0.17/src/caffe/solver.cpp#L393), such as resnet50_score1_score2_score3.caffemodel. Later after finished training, if you use caffe test -model ... -weights ... to test same resnet50_score1_score2_score3.caffemodel. It's a bit lower accuracy then reported during train/val.

drnikolaev commented 6 years ago

@1duo could you post your prototxt file and log please?