How to deploy a trained classification model with exactly the same accuracy?

[zchrissirhcz]

Hi, I'm new to DIGITS and like its WebUI.

I trained a classification model with AlexNet and Kaggle's cats and dogs dataset. By altering the learning rate, my training curves show that after the 1st epoch the validation's accuracy is nearly 100% and keeps that until the last epoch (10-th).

Then I can test a single image via the "Classify One Image" via my local DIGITS web page, and get its probabilities to each class(dog, cat).

However, I'd like to use the trained model in other condition, not on DIGITS. For example, classification is just one component in my whole program. So I'd like to get the exactly the same output in my own code. I tried to use pycaffe to load that trained model and do test on the whole epoch of my validation images. However, the accuracy is low (~50% vs ~100%). So I just wondering how to get it via pycaffe.

(Note, I transformed the mean.binaryproto to mean.npy, but still ~50% accuracy)

My code is like this:

#!/usr/bin/env python2
# coding: utf-8

from __future__ import print_function
import _init_paths
import os
import sys
import numpy as np
import caffe
import cv2

import skimage.io
from PIL import Image

caffe.set_device(0)
caffe.set_mode_gpu()

pre = '/opt/work/digits/digits/jobs/20180412-165330-85a7'
prototxt = os.path.join(pre, 'deploy.prototxt')
caffemodel = os.path.join(pre, 'snapshot_iter_1000.caffemodel')
MEAN_FILE='/home/chris/Downloads/20180412-165330-85a7_epoch_10.0/mean.binaryproto'
mean_blob = caffe.proto.caffe_pb2.BlobProto()
mean_blob.ParseFromString(open(MEAN_FILE, 'rb').read())
mean_npy = caffe.io.blobproto_to_array(mean_blob)
mean_npy = mean_npy[0]
npy_pth = '/home/chris/Downloads/20180412-165330-85a7_epoch_10.0/mean.npy'
np.save(npy_pth, mean_npy)

mean_npy = np.load(npy_pth)
mean = mean_npy.mean(1).mean(1)

net = caffe.Net(prototxt, caffemodel, caffe.TEST)

transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_transpose('data', (2, 0, 1))
transformer.set_mean('data', mean)
transformer.set_raw_scale('data', 255)
transformer.set_channel_swap('data', (2,1,0))

dbpre = '/opt/work/lesion/data/catsdogs'
split_txt = os.path.join(dbpre, 'splits', 'val.txt')
fin = open(split_txt)
im_names = []
im_labels = []
for line in fin.readlines():
    t = line.rstrip('\n').split(' ')
    im_names.append(t[0])
    im_labels.append(int(t[1]))
fin.close()

labels=['cat', 'dog']
num_true = 0
num_tot = 0

for i, im_name in enumerate(im_names):
    im_pth = os.path.join(dbpre, 'images', im_name)
    im = caffe.io.load_image(im_pth)

    net.blobs['data'].data[...] = transformer.preprocess('data', im)
    out = net.forward()

    prob = net.blobs['softmax'].data[0].flatten()
    order = prob.argsort()[-1]

    if order == im_labels[i]:
        num_true += 1
    num_tot += 1

    predicted = labels[order]

    print('testing {:s}, current accuracy is {:.2f}%, predicted as {:s}'.format(im_name, num_true*1.0/num_tot, predicted))

Anyone can help me out? Thanks.

[MaximumProgrammer]

On thing is just that DIGITS uses it's own pre processing for resizing, here is something about it: https://github.com/NVIDIA/DIGITS/issues/1369

You have to change the size before doing the database in DIGITS and before training it, it means change the sizes of the final images before doing the database or change the code of DIGITS for resizing the images because DIGITS uses scipy.image.resize. Ok if you only want to use Python than it's ok but there is now corresponding possibility in c++ for resizing images.

[zchrissirhcz]

@ZeroCool11000 Thanks for your reminding. Actually I would like to see curve plots during training, so I find DIGITS and use it. I personnally like the resizing step directly handle by myself or in a relatively easy seeing manner. So now I change to use VisualDL, which is like a Tensorboard tool for many frameworks including Caffe, and this plotting tool only handle my extracted scalar data to plot without changing them.