search

jessemelpolio / Faster_RCNN_for_DOTA

Code used for training Faster R-CNN on DOTA
https://arxiv.org/abs/1711.10398
Apache License 2.0
336 stars 102 forks source link

libpng error: IDAT: CRC error Exception in thread Thread-1: TypeError: 'NoneType' object has no attribute '__getitem__' AttributeError: 'NoneType' object has no attribute 'shape' #21

Open Jnyle opened 6 years ago

Jnyle commented 6 years ago

Firstly, my terminal outputs are like this: libpng error: IDAT: CRC error Exception in thread Thread-1: Traceback (most recent call last): File "/home/wh/anaconda2/envs/dy/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/home/wh/anaconda2/envs/dy/lib/python2.7/threading.py", line 754, in run self.target(*self.args, **self.kwargs) File "experiments/faster_rcnn/../../faster_rcnn/../lib/utils/PrefetchingIter.py", line 60, in prefetch_func self.next_batch[i] = self.iters[i].next() File "experiments/faster_rcnn/../../faster_rcnn/core/loader.py", line 701, in next self.get_batch_individual() File "experiments/faster_rcnn/../../faster_rcnn/core/loader.py", line 808, in get_batch_individual rst.append(self.parfetch(iroidb)) File "experiments/faster_rcnn/../../faster_rcnn/core/loader.py", line 816, in parfetch data, label = get_rpn_batch_quadrangle(iroidb, self.cfg) File "experiments/faster_rcnn/../../faster_rcnn/../lib/rpn/rpn.py", line 91, in get_rpn_batch_quadrangle imgs, roidb = get_image_quadrangle_bboxes(roidb, cfg) File "experiments/faster_rcnn/../../faster_rcnn/../lib/utils/image.py", line 61, in get_image_quadrangle_bboxes im = im[:, ::-1, :] TypeError: 'NoneType' object has no attribute 'getitem__'

and sometimes like this:

Exception in thread Thread-1: Traceback (most recent call last): File "/home/wh/anaconda2/envs/dy/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/home/wh/anaconda2/envs/dy/lib/python2.7/threading.py", line 754, in run self.target(*self.args, **self.__kwargs) File "experiments/faster_rcnn/../../faster_rcnn/../lib/utils/PrefetchingIter.py", line 61, in prefetch_func self.next_batch[i] = self.iters[i].next() File "experiments/faster_rcnn/../../faster_rcnn/core/loader.py", line 701, in next self.get_batch_individual() File "experiments/faster_rcnn/../../faster_rcnn/core/loader.py", line 808, in get_batch_individual rst.append(self.parfetch(iroidb)) File "experiments/faster_rcnn/../../faster_rcnn/core/loader.py", line 816, in parfetch data, label = get_rpn_batch_quadrangle(iroidb, self.cfg) File "experiments/faster_rcnn/../../faster_rcnn/../lib/rpn/rpn.py", line 91, in get_rpn_batch_quadrangle imgs, roidb = get_image_quadrangle_bboxes(roidb, cfg) File "experiments/faster_rcnn/../../faster_rcnn/../lib/utils/image.py", line 66, in get_image_quadrangle_bboxes im, im_scale = resize(im, target_size, max_size, stride=config.network.IMAGE_STRIDE) File "experiments/faster_rcnn/../../faster_rcnn/../lib/utils/image.py", line 219, in resize im_shape = im.shape AttributeError: 'NoneType' object has no attribute 'shape' I'm sure that my images' format is .png . And my log is as followed: 2018-10-12 20:21:08,725 training config:{'CLASS_AGNOSTIC': False, 'MXNET_VERSION': 'mxnet', 'RESIZE_TO_FIX_SIZE': True, 'SCALES': [(1024, 1024)], 'TEST': {'BATCH_IMAGES': 1, 'CXX_PROPOSAL': False, 'DO_MULTISCALE_TEST': False, 'HAS_RPN': True, 'MULTISCALE': [1.0, 1.2, 1.4, 1.6], 'NMS': 0.3, 'PROPOSAL_MIN_SIZE': 0, 'PROPOSAL_NMS_THRESH': 0.7, 'PROPOSAL_POST_NMS_TOP_N': 2000, 'PROPOSAL_PRE_NMS_TOP_N': 20000, 'RPN_MIN_SIZE': 0, 'RPN_NMS_THRESH': 0.7, 'RPN_POST_NMS_TOP_N': 300, 'RPN_PRE_NMS_TOP_N': 6000, 'max_per_image': 300, 'save_img_path': '/home/wh/Faster_RCNN_for_DOTA/data/vis', 'test_epoch': 59}, 'TRAIN': {'ALTERNATE': {'RCNN_BATCH_IMAGES': 0, 'RPN_BATCH_IMAGES': 0, 'rfcn1_epoch': 0, 'rfcn1_lr': 0, 'rfcn1_lr_step': '', 'rfcn2_epoch': 0, 'rfcn2_lr': 0, 'rfcn2_lr_step': '', 'rpn1_epoch': 0, 'rpn1_lr': 0, 'rpn1_lr_step': '', 'rpn2_epoch': 0, 'rpn2_lr': 0, 'rpn2_lr_step': '', 'rpn3_epoch': 0, 'rpn3_lr': 0, 'rpn3_lr_step': ''}, 'ASPECT_GROUPING': True, 'BATCH_IMAGES': 1, 'BATCH_ROIS': 128, 'BATCH_ROIS_OHEM': 128, 'BBOX_MEANS': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 'BBOX_NORMALIZATION_PRECOMPUTED': False, 'BBOX_REGRESSION_THRESH': 0.5, 'BBOX_STDS': [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], 'BBOX_WEIGHTS': array([1., 1., 1., 1., 1., 1., 1., 1.]), 'BG_THRESH_HI': 0.5, 'BG_THRESH_LO': 0.1, 'CXX_PROPOSAL': False, 'ENABLE_OHEM': True, 'END2END': True, 'FG_FRACTION': 0.25, 'FG_THRESH': 0.5, 'FLIP': True, 'RESUME': False, 'RPN_BATCH_SIZE': 256, 'RPN_BBOX_WEIGHTS': [1.0, 1.0, 1.0, 1.0], 'RPN_CLOBBER_POSITIVES': False, 'RPN_FG_FRACTION': 0.5, 'RPN_MIN_SIZE': 0, 'RPN_NEGATIVE_OVERLAP': 0.3, 'RPN_NMS_THRESH': 0.7, 'RPN_POSITIVE_OVERLAP': 0.7, 'RPN_POSITIVE_WEIGHT': -1.0, 'RPN_POST_NMS_TOP_N': 300, 'RPN_PRE_NMS_TOP_N': 6000, 'SHUFFLE': True, 'begin_epoch': 0, 'end_epoch': 60, 'lr': 0.0005, 'lr_factor': 0.1, 'lr_step': '45,52', 'model_prefix': 'rcnn_DOTA_quadrangle', 'momentum': 0.9, 'warmup': True, 'warmup_lr': 5e-05, 'warmup_step': 1000, 'wd': 0.0005}, 'dataset': {'NUM_CLASSES': 16, 'dataset': 'DOTA_oriented', 'dataset_path': '/home/wh/Faster_RCNN_for_DOTA/data', 'image_set': 'train', 'proposal': 'rpn', 'root_path': '/home/wh/Faster_RCNN_for_DOTA/data', 'test_image_set': 'test'}, 'default': {'frequent': 100, 'kvstore': 'device'}, 'gpus': '0', 'network': {'ANCHOR_RATIOS': [0.5, 1, 2], 'ANCHOR_SCALES': [8, 16, 32], 'FIXED_PARAMS': ['conv1', 'bn_conv1', 'res2', 'bn2', 'gamma', 'beta'], 'FIXED_PARAMS_SHARED': ['conv1', 'bn_conv1', 'res2', 'bn2', 'res3', 'bn3', 'res4', 'bn4', 'gamma', 'beta'], 'IMAGE_STRIDE': 0, 'NUM_ANCHORS': 9, 'PIXEL_MEANS': array([103.06, 115.9 , 123.15]), 'RCNN_FEAT_STRIDE': 16, 'RPN_FEAT_STRIDE': 16, 'pretrained': './model/pretrained_model/resnet_v1_101', 'pretrained_epoch': 0}, 'output_path': './output/rcnn/DOTA_quadrangle', 'symbol': 'resnet_v1_101_rcnn_quadrangle'}

2018-10-12 20:21:11,505 bucketing: data "gt_boxes" has a shape (1L, 386L, 9L), which is larger than already allocated shape (1L, 100L, 9L). Need to re-allocate. Consider putting default_bucket_key to be the bucket taking the largest input for better memory sharing. 2018-10-12 20:21:19,930 bucketing: data "gt_boxes" has a shape (1L, 597L, 9L), which is larger than already allocated shape (1L, 386L, 9L). Need to re-allocate. Consider putting default_bucket_key to be the bucket taking the largest input for better memory sharing. 2018-10-12 20:21:50,888 training config:{'CLASS_AGNOSTIC': False, 'MXNET_VERSION': 'mxnet', 'RESIZE_TO_FIX_SIZE': True, 'SCALES': [(1024, 1024)], 'TEST': {'BATCH_IMAGES': 1, 'CXX_PROPOSAL': False, 'DO_MULTISCALE_TEST': False, 'HAS_RPN': True, 'MULTISCALE': [1.0, 1.2, 1.4, 1.6], 'NMS': 0.3, 'PROPOSAL_MIN_SIZE': 0, 'PROPOSAL_NMS_THRESH': 0.7, 'PROPOSAL_POST_NMS_TOP_N': 2000, 'PROPOSAL_PRE_NMS_TOP_N': 20000, 'RPN_MIN_SIZE': 0, 'RPN_NMS_THRESH': 0.7, 'RPN_POST_NMS_TOP_N': 300, 'RPN_PRE_NMS_TOP_N': 6000, 'max_per_image': 300, 'save_img_path': '/home/wh/Faster_RCNN_for_DOTA/data/vis', 'test_epoch': 59}, 'TRAIN': {'ALTERNATE': {'RCNN_BATCH_IMAGES': 0, 'RPN_BATCH_IMAGES': 0, 'rfcn1_epoch': 0, 'rfcn1_lr': 0, 'rfcn1_lr_step': '', 'rfcn2_epoch': 0, 'rfcn2_lr': 0, 'rfcn2_lr_step': '', 'rpn1_epoch': 0, 'rpn1_lr': 0, 'rpn1_lr_step': '', 'rpn2_epoch': 0, 'rpn2_lr': 0, 'rpn2_lr_step': '', 'rpn3_epoch': 0, 'rpn3_lr': 0, 'rpn3_lr_step': ''}, 'ASPECT_GROUPING': True, 'BATCH_IMAGES': 1, 'BATCH_ROIS': 128, 'BATCH_ROIS_OHEM': 128, 'BBOX_MEANS': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 'BBOX_NORMALIZATION_PRECOMPUTED': False, 'BBOX_REGRESSION_THRESH': 0.5, 'BBOX_STDS': [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], 'BBOX_WEIGHTS': array([1., 1., 1., 1., 1., 1., 1., 1.]), 'BG_THRESH_HI': 0.5, 'BG_THRESH_LO': 0.1, 'CXX_PROPOSAL': False, 'ENABLE_OHEM': True, 'END2END': True, 'FG_FRACTION': 0.25, 'FG_THRESH': 0.5, 'FLIP': True, 'RESUME': False, 'RPN_BATCH_SIZE': 256, 'RPN_BBOX_WEIGHTS': [1.0, 1.0, 1.0, 1.0], 'RPN_CLOBBER_POSITIVES': False, 'RPN_FG_FRACTION': 0.5, 'RPN_MIN_SIZE': 0, 'RPN_NEGATIVE_OVERLAP': 0.3, 'RPN_NMS_THRESH': 0.7, 'RPN_POSITIVE_OVERLAP': 0.7, 'RPN_POSITIVE_WEIGHT': -1.0, 'RPN_POST_NMS_TOP_N': 300, 'RPN_PRE_NMS_TOP_N': 6000, 'SHUFFLE': True, 'begin_epoch': 0, 'end_epoch': 60, 'lr': 0.0005, 'lr_factor': 0.1, 'lr_step': '45,52', 'model_prefix': 'rcnn_DOTA_quadrangle', 'momentum': 0.9, 'warmup': True, 'warmup_lr': 5e-05, 'warmup_step': 1000, 'wd': 0.0005}, 'dataset': {'NUM_CLASSES': 16, 'dataset': 'DOTA_oriented', 'dataset_path': '/home/wh/Faster_RCNN_for_DOTA/data', 'image_set': 'train', 'proposal': 'rpn', 'root_path': '/home/wh/Faster_RCNN_for_DOTA/data', 'test_image_set': 'test'}, 'default': {'frequent': 100, 'kvstore': 'device'}, 'gpus': '0', 'network': {'ANCHOR_RATIOS': [0.5, 1, 2], 'ANCHOR_SCALES': [8, 16, 32], 'FIXED_PARAMS': ['conv1', 'bn_conv1', 'res2', 'bn2', 'gamma', 'beta'], 'FIXED_PARAMS_SHARED': ['conv1', 'bn_conv1', 'res2', 'bn2', 'res3', 'bn3', 'res4', 'bn4', 'gamma', 'beta'], 'IMAGE_STRIDE': 0, 'NUM_ANCHORS': 9, 'PIXEL_MEANS': array([103.06, 115.9 , 123.15]), 'RCNN_FEAT_STRIDE': 16, 'RPN_FEAT_STRIDE': 16, 'pretrained': './model/pretrained_model/resnet_v1_101', 'pretrained_epoch': 0}, 'output_path': './output/rcnn/DOTA_quadrangle', 'symbol': 'resnet_v1_101_rcnn_quadrangle'}

2018-10-12 20:21:55,379 bucketing: data "gt_boxes" has a shape (1L, 263L, 9L), which is larger than already allocated shape (1L, 100L, 9L). Need to re-allocate. Consider putting default_bucket_key to be the bucket taking the largest input for better memory sharing. 2018-10-12 20:21:56,718 bucketing: data "gt_boxes" has a shape (1L, 295L, 9L), which is larger than already allocated shape (1L, 263L, 9L). Need to re-allocate. Consider putting default_bucket_key to be the bucket taking the largest input for better memory sharing. 2018-10-12 20:22:08,772 bucketing: data "gt_boxes" has a shape (1L, 1028L, 9L), which is larger than already allocated shape (1L, 295L, 9L). Need to re-allocate. Consider putting default_bucket_key to be the bucket taking the largest input for better memory sharing. 2018-10-12 20:22:19,004 Epoch[0] Batch [100] Speed: 3.89 samples/sec Train-RPNAcc=0.838219, RPNLogLoss=0.442614, RPNL1Loss=0.823888, RCNNAcc=0.750155, RCNNLogLoss=2.288598, RCNNL1Loss=0.208822,

So, what's wrong?

jessemelpolio commented 6 years ago

I think the problem should lie in data processing. Have a look. It might not be the format problem. Maybe some of your images are corrupted or the image path is not pointing to an existing image. I don't quite know. I guess it should be the data problem.

Jnyle commented 6 years ago

There are some wrong images, but I check the other images and find no problem and there are still the same problems when training:

TypeError: 'NoneType' object has no attribute 'getitem'

The image name was output in the screen and I check that it can be opened.