When I train the model using my Images. Command terminated by signal 9

[LingZhang0215]

(tf) ling@ubuntu:~/Desktop/tf-faster-rcnn-master$ ./experiments/scripts/train_faster_rcnn.sh 0 pascal_voc vgg16

• set -e
• export PYTHONUNBUFFERED=True
• PYTHONUNBUFFERED=True
• GPU_ID=0
• DATASET=pascal_voc
• NET=vgg16
• array=($@)
• len=3
• EXTRA_ARGS=
• EXTRA_ARGS_SLUG=
• case ${DATASET} in
• TRAIN_IMDB=voc_2007_trainval
• TEST_IMDB=voc_2007_test
• STEPSIZE='[50000]'
• ITERS=70000
• ANCHORS='[8,16,32]'
• RATIOS='[0.5,1,2]' ++ date +%Y-%m-%d_%H-%M-%S
• LOG=experiments/logs/vgg16_voc_2007_trainval__vgg16.txt.2020-04-27_12-21-01
• exec ++ tee -a experiments/logs/vgg16_voc_2007_trainval__vgg16.txt.2020-04-27_12-21-01
• echo Logging output to experiments/logs/vgg16_voc_2007_trainvalvgg16.txt.2020-04-27_12-21-01 Logging output to experiments/logs/vgg16_voc_2007_trainvalvgg16.txt.2020-04-27_12-21-01
• set +x
• '[' '!' -f output/vgg16/voc_2007_trainval/default/vgg16_faster_rcnn_iter_70000.ckpt.index ']'
• [[ ! -z '' ]]
• CUDA_VISIBLE_DEVICES=0
• time python ./tools/trainval_net.py --weight data/imagenet_weights/vgg16.ckpt --imdb voc_2007_trainval --imdbval voc_2007_test --iters 70000 --cfg experiments/cfgs/vgg16.yml --net vgg16 --set ANCHOR_SCALES '[8,16,32]' ANCHOR_RATIOS '[0.5,1,2]' TRAIN.STEPSIZE '[50000]' /home/ling/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /home/ling/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /home/ling/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /home/ling/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /home/ling/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /home/ling/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) Called with args: Namespace(cfg_file='experiments/cfgs/vgg16.yml', imdb_name='voc_2007_trainval', imdbval_name='voc_2007_test', max_iters=70000, net='vgg16', set_cfgs=['ANCHOR_SCALES', '[8,16,32]', 'ANCHOR_RATIOS', '[0.5,1,2]', 'TRAIN.STEPSIZE', '[50000]'], tag=None, weight='data/imagenet_weights/vgg16.ckpt') /home/ling/Desktop/tf-faster-rcnn-master/tools/../lib/model/config.py:362: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. yaml_cfg = edict(yaml.load(f)) Using config: {'ANCHOR_RATIOS': [0.5, 1, 2], 'ANCHOR_SCALES': [8, 16, 32], 'DATA_DIR': '/home/ling/Desktop/tf-faster-rcnn-master/data', 'EXP_DIR': 'vgg16', 'MATLAB': 'matlab', 'MOBILENET': {'DEPTH_MULTIPLIER': 1.0, 'FIXED_LAYERS': 5, 'REGU_DEPTH': False, 'WEIGHT_DECAY': 4e-05}, 'PIXEL_MEANS': array([[[102.9801, 115.9465, 122.7717]]]), 'POOLING_MODE': 'crop', 'POOLING_SIZE': 7, 'RESNET': {'FIXED_BLOCKS': 1, 'MAX_POOL': False}, 'RNG_SEED': 3, 'ROOT_DIR': '/home/ling/Desktop/tf-faster-rcnn-master', 'RPN_CHANNELS': 512, 'TEST': {'BBOX_REG': True, 'HAS_RPN': True, 'MAX_SIZE': 1000, 'MODE': 'nms', 'NMS': 0.3, 'PROPOSAL_METHOD': 'gt', 'RPN_NMS_THRESH': 0.7, 'RPN_POST_NMS_TOP_N': 300, 'RPN_PRE_NMS_TOP_N': 6000, 'RPN_TOP_N': 5000, 'SCALES': [600], 'SVM': False}, 'TRAIN': {'ASPECT_GROUPING': False, 'BATCH_SIZE': 256, 'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0], 'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0], 'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2], 'BBOX_NORMALIZE_TARGETS': True, 'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': True, 'BBOX_REG': True, 'BBOX_THRESH': 0.5, 'BG_THRESH_HI': 0.5, 'BG_THRESH_LO': 0.0, 'BIAS_DECAY': False, 'DISPLAY': 20, 'DOUBLE_BIAS': True, 'FG_FRACTION': 0.25, 'FG_THRESH': 0.5, 'GAMMA': 0.1, 'HAS_RPN': True, 'IMS_PER_BATCH': 1, 'LEARNING_RATE': 0.001, 'MAX_SIZE': 1000, 'MOMENTUM': 0.9, 'PROPOSAL_METHOD': 'gt', 'RPN_BATCHSIZE': 256, 'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0], 'RPN_CLOBBER_POSITIVES': False, 'RPN_FG_FRACTION': 0.5, 'RPN_NEGATIVE_OVERLAP': 0.3, 'RPN_NMS_THRESH': 0.7, 'RPN_POSITIVE_OVERLAP': 0.7, 'RPN_POSITIVE_WEIGHT': -1.0, 'RPN_POST_NMS_TOP_N': 2000, 'RPN_PRE_NMS_TOP_N': 12000, 'SCALES': [600], 'SNAPSHOT_ITERS': 5000, 'SNAPSHOT_KEPT': 3, 'SNAPSHOT_PREFIX': 'vgg16_faster_rcnn', 'STEPSIZE': [50000], 'SUMMARY_INTERVAL': 180, 'TRUNCATED': False, 'USE_ALL_GT': True, 'USE_FLIPPED': True, 'USE_GT': False, 'WEIGHT_DECAY': 0.0001}, 'USE_E2E_TF': True, 'USE_GPU_NMS': True} Loaded dataset voc_2007_trainval for training Set proposal method: gt Appending horizontally-flipped training examples... wrote gt roidb to /home/ling/Desktop/tf-faster-rcnn-master/data/cache/voc_2007_trainval_gt_roidb.pkl done Preparing training data... done 30 roidb entries Output will be saved to /home/ling/Desktop/tf-faster-rcnn-master/output/vgg16/voc_2007_trainval/default TensorFlow summaries will be saved to /home/ling/Desktop/tf-faster-rcnn-master/tensorboard/vgg16/voc_2007_trainval/default Loaded dataset voc_2007_test for training Set proposal method: gt Preparing training data... wrote gt roidb to /home/ling/Desktop/tf-faster-rcnn-master/data/cache/voc_2007_test_gt_roidb.pkl done 14 validation roidb entries Filtered 0 roidb entries: 30 -> 30 Filtered 0 roidb entries: 14 -> 14 2020-04-27 12:21:04.801677: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA 2020-04-27 12:21:04.828290: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance. Solving... /home/ling/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " Loading initial model weights from data/imagenet_weights/vgg16.ckpt Variables restored: vgg_16/conv1/conv1_1/biases:0 Variables restored: vgg_16/conv1/conv1_2/weights:0 Variables restored: vgg_16/conv1/conv1_2/biases:0 Variables restored: vgg_16/conv2/conv2_1/weights:0 Variables restored: vgg_16/conv2/conv2_1/biases:0 Variables restored: vgg_16/conv2/conv2_2/weights:0 Variables restored: vgg_16/conv2/conv2_2/biases:0 Variables restored: vgg_16/conv3/conv3_1/weights:0 Variables restored: vgg_16/conv3/conv3_1/biases:0 Variables restored: vgg_16/conv3/conv3_2/weights:0 Variables restored: vgg_16/conv3/conv3_2/biases:0 Variables restored: vgg_16/conv3/conv3_3/weights:0 Variables restored: vgg_16/conv3/conv3_3/biases:0 Variables restored: vgg_16/conv4/conv4_1/weights:0 Variables restored: vgg_16/conv4/conv4_1/biases:0 Variables restored: vgg_16/conv4/conv4_2/weights:0 Variables restored: vgg_16/conv4/conv4_2/biases:0 Variables restored: vgg_16/conv4/conv4_3/weights:0 Variables restored: vgg_16/conv4/conv4_3/biases:0 Variables restored: vgg_16/conv5/conv5_1/weights:0 Variables restored: vgg_16/conv5/conv5_1/biases:0 Variables restored: vgg_16/conv5/conv5_2/weights:0 Variables restored: vgg_16/conv5/conv5_2/biases:0 Variables restored: vgg_16/conv5/conv5_3/weights:0 Variables restored: vgg_16/conv5/conv5_3/biases:0 Variables restored: vgg_16/fc6/biases:0 Variables restored: vgg_16/fc7/biases:0 Loaded. Fix VGG16 layers.. Command terminated by signal 9 6.69user 3.21system 0:22.24elapsed 44%CPU (0avgtext+0avgdata 2786176maxresident)k 1026832inputs+1736outputs (390major+909294minor)pagefaults 0swaps

So signal 9 means what kind of problem?

[hyx1024]

Did you finally solve it?

[LingZhang0215]

Did you finally solve it?

Yea, I just do all the steps from the beginning again. Then no more error happened.