Poor performance using the features extracted from caffe

[qixianbiao]

Hello, When I try to use Caffe to implement two papers, I met some problems. The papers are 'CNN features off-the-shelf: An ostounding baseline for recognition', 'Decribeing texture in the wild'. In their experiments, they just use the alexnet to extract 4096-D CNN feature, and then use an SVM to conduct classification. I follow the feature_extraction page and extract features for the images in the dataset.

  I success to run the experiments but get much lower performance, for example in texture classification, they reported 60% but I only get about 40%, on oxford flower, they reported around 80% but I only obtained 45%. 

  BTW, the performance for the first paper is reported according to Overfeat, and the second paper is based on Decaf. The protocol file is below.

BR Xianbiao

name: "CaffeNet" layers { name: "data" type: IMAGE_DATA top: "data" top: "label" image_dataparam { source: "/home/**/DeepLearning/caffe-master/OxfordFlower/file_list.txt" meanfile: "/home/**/DeepLearning/caffe-master/data/ilsvrc12/imagenet_mean.binaryproto" batch_size: 1 mirror: false crop_size: 227 new_height: 256 new_width: 256 } }

[GregAtHeron]

Typically you should contact the authors first, to walk them through what you tried first.

[qixianbiao]

@GregAtHeron, I just finish using OverFeat to implement both papers and can get the reported performance. I totally believe I misuse caffe. But, until now, I still don't figure out the reason.

[shelhamer]

OverFeat has two distinct models ("fast" and "accurate") from AlexNet and CaffeNet. Consult the OverFeat paper and documentation for some details of their model architecture. Note in principle you could replicate their model in another framework like Caffe, pylearn2, etc. but no one has done so yet. It seems it is more delicate to train than AlexNet or CaffeNet.

See the architectures at http://cilvr.nyu.edu/doku.php?id=software:overfeat:start and note the differences. There is no reason for the features to perform the same; although the dimension is different, they are different embeddings.

On Fri, Jul 11, 2014 at 12:46 PM, qixianbiao notifications@github.com wrote:

@GregAtHeron https://github.com/GregAtHeron, I just finish using OverFeat to implement both papers and can get the reported performance. I totally believe I misuse caffe. But, until now, I still don't figure out the reason.

— Reply to this email directly or view it on GitHub https://github.com/BVLC/caffe/issues/642#issuecomment-48717103.

[HoldenCaulfieldRye]

@qixianbiao I am also interested in reproducing the results of the 'CNN features off the shelf' paper. Are you still not able to reach the reported accuracy with caffe?