Trained resnet is good performance ,but run the demo.py it si bad.

[abner2015]

Trained resnet is good performance ,but run the demo.py it is bad. I just changed the demo.py as : (1)from rcnn.symbol import get_resnet_test (2)symbol = get_resnet_test(num_classes=config.NUM_CLASSES, num_anchors=config.NUM_ANCHORS)

Then run: python demo.py --prefix ./model/e2e --epoch 10 --image 000001.jpg

Anyone can tell me why? How can i do?

[ijkguo]

Check the differences between test and demo. I think it is pixel means. Please report your observation or solution to help others. Thanks.

[abner2015]

The image is .bmp format, and is 8bit gray image.

# Trian : INFO:root:Epoch[0] Train-RPNAcc=0.862934 INFO:root:Epoch[0] Train-RPNLogLoss=0.277668 INFO:root:Epoch[0] Train-RPNL1Loss=1.381117 INFO:root:Epoch[0] Train-RCNNAcc=0.987370 INFO:root:Epoch[0] Train-RCNNLogLoss=0.074089 INFO:root:Epoch[0] Train-RCNNL1Loss=0.431803 INFO:root:Epoch[0] Time cost=78.749

# Test :

CMD:python test.py --network resnet --gpu 0

I had execute the cmd many times ERROR: [11:35:48] /mxnet/dmlc-core/include/dmlc/./logging.h:300: [11:35:48] src/engine/./threaded_engine.h:336: [11:35:48] src/operator/proposal.cu:496: Check failed: error == cudaSuccess (77 vs. 0) an illegal memory access was encountered

All result as the follow: results saved to ./reanetResult/000090.bmp class ---- [[x1, x2, y1, y2, confidence]]

That mean no detection object!

[ijkguo]

I have seen this before but could not reproduce and fix it. Would you please share more details?

[abner2015]

@precedenceguo But, sometime test is Namespace(dataset='PascalVOC', dataset_path='data/VOCdevkit', epoch=10, gpu=0, has_rpn=True, image_set='2007_test', network='resnet', prefix='model/e2e', proposal='rpn', root_path='data', shuffle=False, thresh=0.001, vis=False) {'ANCHOR_RATIOS': [0.5, 1, 2], 'ANCHOR_SCALES': [8, 16, 32], 'FIXED_PARAMS': ['conv0', 'stage1', 'gamma', 'beta'], 'FIXED_PARAMS_SHARED': ['conv0', 'stage1', 'stage2', 'stage3', 'gamma', 'beta'], 'IMAGE_STRIDE': 0, 'NUM_ANCHORS': 9, 'NUM_CLASSES': 2, 'PIXEL_MEANS': array([0, 0, 0]), 'RCNN_FEAT_STRIDE': 16, 'RPN_FEAT_STRIDE': 16, 'SCALES': [(600, 1000)], 'TEST': {'BATCH_IMAGES': 1, 'CXX_PROPOSAL': True, 'HAS_RPN': True, 'NMS': 0.3, 'PROPOSAL_MIN_SIZE': 16, 'PROPOSAL_NMS_THRESH': 0.7, 'PROPOSAL_POST_NMS_TOP_N': 2000, 'PROPOSAL_PRE_NMS_TOP_N': 20000, 'RPN_MIN_SIZE': 16, 'RPN_NMS_THRESH': 0.7, 'RPN_POST_NMS_TOP_N': 300, 'RPN_PRE_NMS_TOP_N': 6000}, 'TRAIN': {'ASPECT_GROUPING': True, 'BATCH_IMAGES': 2, 'BATCH_ROIS': 128, 'BBOX_MEANS': [0.0, 0.0, 0.0, 0.0], 'BBOX_NORMALIZATION_PRECOMPUTED': False, 'BBOX_REGRESSION_THRESH': 0.5, 'BBOX_STDS': [0.1, 0.1, 0.2, 0.2], 'BBOX_WEIGHTS': array([ 1., 1., 1., 1.]), 'BG_THRESH_HI': 0.5, 'BG_THRESH_LO': 0.0, 'CXX_PROPOSAL': True, 'END2END': False, 'FG_FRACTION': 0.25, 'FG_THRESH': 0.5, 'RPN_BATCH_SIZE': 256, 'RPN_BBOX_WEIGHTS': [1.0, 1.0, 1.0, 1.0], 'RPN_CLOBBER_POSITIVES': False, 'RPN_FG_FRACTION': 0.5, 'RPN_MIN_SIZE': 16, 'RPN_NEGATIVE_OVERLAP': 0.3, 'RPN_NMS_THRESH': 0.7, 'RPN_POSITIVE_OVERLAP': 0.7, 'RPN_POSITIVE_WEIGHT': -1.0, 'RPN_POST_NMS_TOP_N': 2000, 'RPN_PRE_NMS_TOP_N': 12000}} dddddddddddddddddddddd num_images 41 voc_2007_test gt roidb loaded from data/cache/voc_2007_test_gt_roidb.pkl /home/root1/.local/lib/python2.7/site-packages/mxnet-0.9.4-py2.7.egg/mxnet/module/base_module.py:64: UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['cls_prob_label']) warnings.warn(msg) [09:58:30] src/operator/./cudnn_convolution-inl.h:55: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable) testing 0/41 data 0.0374s net 2.0888s post 0.0014s testing 1/41 data 0.0862s net 0.1577s post 0.0012s testing 2/41 data 0.0878s net 0.1565s post 0.0009s testing 3/41 data 0.0850s net 0.1569s post 0.0020s testing 4/41 data 0.0734s net 0.1558s post 0.0026s testing 5/41 data 0.0734s net 0.1568s post 0.0013s testing 6/41 data 0.0685s net 0.1562s post 0.0013s testing 7/41 data 0.0817s net 0.1569s post 0.0022s testing 8/41 data 0.0745s net 0.1564s post 0.0017s testing 9/41 data 0.0744s net 0.1569s post 0.0018s testing 10/41 data 0.0866s net 0.1620s post 0.0027s testing 11/41 data 0.0965s net 0.1560s post 0.0026s testing 12/41 data 0.0668s net 0.1576s post 0.0018s testing 13/41 data 0.0683s net 0.1558s post 0.0015s testing 14/41 data 0.0702s net 0.1565s post 0.0022s testing 15/41 data 0.0720s net 0.1586s post 0.0019s testing 16/41 data 0.0709s net 0.1554s post 0.0023s testing 17/41 data 0.0801s net 0.1582s post 0.0025s testing 18/41 data 0.0644s net 0.1563s post 0.0008s testing 19/41 data 0.0752s net 0.1572s post 0.0018s testing 20/41 data 0.0658s net 0.1575s post 0.0014s testing 21/41 data 0.0799s net 0.1564s post 0.0017s testing 22/41 data 0.0700s net 0.1568s post 0.0025s testing 23/41 data 0.1204s net 0.1571s post 0.0023s testing 24/41 data 0.0628s net 0.1580s post 0.0024s testing 25/41 data 0.0851s net 0.1567s post 0.0017s testing 26/41 data 0.0643s net 0.1561s post 0.0019s testing 27/41 data 0.0712s net 0.1571s post 0.0015s testing 28/41 data 0.0652s net 0.1557s post 0.0017s testing 29/41 data 0.0765s net 0.1566s post 0.0016s testing 30/41 data 0.0762s net 0.1573s post 0.0016s testing 31/41 data 0.0695s net 0.1549s post 0.0016s testing 32/41 data 0.0803s net 0.1565s post 0.0018s testing 33/41 data 0.0627s net 0.1567s post 0.0020s testing 34/41 data 0.0783s net 0.1557s post 0.0019s testing 35/41 data 0.0786s net 0.1566s post 0.0019s testing 36/41 data 0.0653s net 0.1559s post 0.0018s testing 37/41 data 0.0716s net 0.1590s post 0.0024s testing 38/41 data 0.0741s net 0.1566s post 0.0018s testing 39/41 data 0.0677s net 0.1556s post 0.0017s testing 40/41 data 0.0636s net 0.1587s post 0.0016s Writing baidian VOC results file VOC07 metric? Y AP for baidian = 0.5838 Mean AP = 0.5838 # Demo.py `import argparse import os import cv2 import mxnet as mx import numpy as np from rcnn.config import config from rcnn.symbol import get_vgg_test, get_vgg_rpn_test from rcnn.symbol import get_resnet_test from rcnn.io.image import resize, transform from rcnn.core.tester import Predictor, im_detect, im_proposal, vis_all_detection, draw_all_detection from rcnn.utils.load_model import load_param from rcnn.processing.nms import py_nms_wrapper, cpu_nms_wrapper, gpu_nms_wrapper

CLASSES = ('background', 'baidian') config.TEST.HAS_RPN = True SHORT_SIDE = config.SCALES[0][0] LONG_SIDE = config.SCALES[0][1] PIXEL_MEANS = config.PIXEL_MEANS DATA_NAMES = ['data', 'im_info'] LABEL_NAMES = ['cls_prob_label'] DATA_SHAPES = [('data', (1, 3, LONG_SIDE, SHORT_SIDE)), ('im_info', (1, 3))] LABEL_SHAPES = None

visualization

CONF_THRESH = 0.7 NMS_THRESH = 0.3 nms = py_nms_wrapper(NMS_THRESH)

def get_net(symbol, prefix, epoch, ctx): arg_params, aux_params = load_param(prefix, epoch, convert=True, ctx=ctx, process=True)

# infer shape
data_shape_dict = dict(DATA_SHAPES)
arg_names, aux_names = symbol.list_arguments(), symbol.list_auxiliary_states()
arg_shape, _, aux_shape = symbol.infer_shape(**data_shape_dict)
arg_shape_dict = dict(zip(arg_names, arg_shape))
aux_shape_dict = dict(zip(aux_names, aux_shape))

# check shapes
for k in symbol.list_arguments():
    if k in data_shape_dict or 'label' in k:
        continue
    assert k in arg_params, k + ' not initialized'
    assert arg_params[k].shape == arg_shape_dict[k], \
        'shape inconsistent for ' + k + ' inferred ' + str(arg_shape_dict[k]) + ' provided ' + str(arg_params[k].shape)
for k in symbol.list_auxiliary_states():
    assert k in aux_params, k + ' not initialized'
    assert aux_params[k].shape == aux_shape_dict[k], \
        'shape inconsistent for ' + k + ' inferred ' + str(aux_shape_dict[k]) + ' provided ' + str(aux_params[k].shape)

predictor = Predictor(symbol, DATA_NAMES, LABEL_NAMES, context=ctx,
                      provide_data=DATA_SHAPES, provide_label=LABEL_SHAPES,
                      arg_params=arg_params, aux_params=aux_params)
return predictor

def generate_batch(im): """ preprocess image, return batch :param im: cv2.imread returns [height, width, channel] in BGR :return: data_batch: MXNet input batch data_names: names in data_batch im_scale: float number """ im_array, im_scale = resize(im, SHORT_SIDE, LONG_SIDE) im_array = transform(im_array, PIXEL_MEANS) im_info = np.array([[im_array.shape[2], im_array.shape[3], im_scale]], dtype=np.float32) data = [mx.nd.array(im_array), mx.nd.array(im_info)] data_shapes = [('data', im_array.shape), ('im_info', im_info.shape)] data_batch = mx.io.DataBatch(data=data, label=None, provide_data=data_shapes, provide_label=None) return data_batch, DATA_NAMES, im_scale

def demo_net(predictor, image_name,file, vis=False): """ generate data_batch -> im_detect -> post process :param predictor: Predictor :param image_name: image name :param vis: will save as a new image if not visualized :return: None """ assert os.path.exists(image_name), image_name + ' not found' im = cv2.imread(image_name) data_batch, data_names, im_scale = generate_batch(im) scores, boxes, data_dict = im_detect(predictor, data_batch, data_names, im_scale)

all_boxes = [[] for _ in CLASSES]
for cls in CLASSES:
    cls_ind = CLASSES.index(cls)
    cls_boxes = boxes[:, 4 * cls_ind:4 * (cls_ind + 1)]
    cls_scores = scores[:, cls_ind, np.newaxis]
    keep = np.where(cls_scores >= CONF_THRESH)[0]
    dets = np.hstack((cls_boxes, cls_scores)).astype(np.float32)[keep, :]
    keep = nms(dets)
    all_boxes[cls_ind] = dets[keep, :]

boxes_this_image = [[]] + [all_boxes[j] for j in range(1, len(CLASSES))]

# print results
print 'class ---- [[x1, x2, y1, y2, confidence]]'
for ind, boxes in enumerate(boxes_this_image):
    if len(boxes) > 0:
        print '---------', CLASSES[ind], '---------'
        print boxes

if vis:
    vis_all_detection(data_dict['data'].asnumpy(), boxes_this_image, CLASSES, im_scale)
else:
    #result_file = image_name.replace('.', '_result.')
    result_file = file.replace('.', '_result.')
    result_file = './reanetResult/' + file
    print 'results saved to %s' % result_file
    im = draw_all_detection(data_dict['data'].asnumpy(), boxes_this_image, CLASSES, im_scale)
    cv2.imwrite(result_file, im)

def parse_args(): parser = argparse.ArgumentParser(description='Demonstrate a Faster R-CNN network') parser.add_argument('--image', help='custom image', type=str) parser.add_argument('--prefix', help='saved model prefix', type=str) parser.add_argument('--epoch', help='epoch of pretrained model', type=int) parser.add_argument('--gpu', help='GPU device to use', default=0, type=int) parser.add_argument('--vis', help='display result', action='store_true') args = parser.parse_args() return args

def main(): args = parse_args() ctx = mx.gpu(args.gpu) symbol = get_resnet_test(num_classes=config.NUM_CLASSES, num_anchors=config.NUM_ANCHORS) predictor = get_net(symbol, args.prefix, args.epoch, ctx) dir = '/home/root1/new/software/mxnet/example/rcnn/test' for root, dirs, files in os.walk(dir): for file in files: image = root + '/' + file print 'image : ', image demo_net(predictor, image, file, args.vis)

if name == 'main': main()`

[ijkguo]

Ok I cannot reproduce it now. If you would like to fix it, you are welcome to submit a pr.

[315386775]

@abner2015 Did you slove the problem?