search

matterport / Mask_RCNN

Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow
Other
24.55k stars 11.69k forks source link

Can I train Mask_Rcnn with no mask json annotation #2532

Open duongnv0499 opened 3 years ago

duongnv0499 commented 3 years ago

Hi, thank for great work my data is a json file in COCO format but it does not include any segmentation annotation :

{"area": 10712, "iscrowd": 0, "image_id": 379, "bbox": [557, 747, 103, 104], "category_id": 4, "id": 3286, "ignore": 0, "segmentation": []}

I got this error when using my data :
` WARNING:root:Fail load requirements file, so using default ones. /usr/local/lib/python3.7/dist-packages/setuptools/dist.py:645: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead % (opt, underscore_opt)) /usr/local/lib/python3.7/dist-packages/setuptools/dist.py:645: UserWarning: Usage of dash-separated 'license-file' will not be supported in future versions. Please use the underscore name 'license_file' instead % (opt, underscore_opt)) /usr/local/lib/python3.7/dist-packages/setuptools/dist.py:645: UserWarning: Usage of dash-separated 'requirements-file' will not be supported in future versions. Please use the underscore name 'requirements_file' instead % (opt, underscore_opt)) running install running bdist_egg running egg_info writing mask_rcnn.egg-info/PKG-INFO writing dependency_links to mask_rcnn.egg-info/dependency_links.txt writing top-level names to mask_rcnn.egg-info/top_level.txt warning: Failed to find the configured license file 'LICENSE' reading manifest file 'mask_rcnn.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'README.md' warning: no files found matching 'LICENSE' writing manifest file 'mask_rcnn.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/visualize.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/config.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/model.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/save_result.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/init.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/utils.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/parallel_model.py -> build/bdist.linux-x86_64/egg/mrcnn byte-compiling build/bdist.linux-x86_64/egg/mrcnn/visualize.py to visualize.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/config.py to config.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/model.py to model.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/save_result.py to save_result.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/init.py to init.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/utils.py to utils.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/parallel_model.py to parallel_model.cpython-37.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying mask_rcnn.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying mask_rcnn.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying mask_rcnn.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying mask_rcnn.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating 'dist/mask_rcnn-2.1-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing mask_rcnn-2.1-py3.7.egg Removing /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg Copying mask_rcnn-2.1-py3.7.egg to /usr/local/lib/python3.7/dist-packages mask-rcnn 2.1 is already the active version in easy-install.pth

Installed /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg Processing dependencies for mask-rcnn==2.1 Finished processing dependencies for mask-rcnn==2.1 [ ] !python3 samples/coco/coco.py train --dataset=/content/Mask_RCNN/dataset/coco/ --model=coco /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) Using TensorFlow backend. WARNING:tensorflow:From samples/coco/coco.py:75: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From samples/coco/coco.py:77: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-04-12 18:34:25.759206: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2021-04-12 18:34:25.763629: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2299995000 Hz 2021-04-12 18:34:25.763909: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55bb3a2b9340 executing computations on platform Host. Devices: 2021-04-12 18:34:25.763948: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , 2021-04-12 18:34:25.766009: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1 2021-04-12 18:34:25.841117: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.842035: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55bb3a2b9500 executing computations on platform CUDA. Devices: 2021-04-12 18:34:25.842072: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): Tesla K80, Compute Capability 3.7 2021-04-12 18:34:25.842262: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.843002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235 pciBusID: 0000:00:04.0 2021-04-12 18:34:25.843364: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2021-04-12 18:34:25.844951: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0 2021-04-12 18:34:25.846810: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0 2021-04-12 18:34:25.847213: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0 2021-04-12 18:34:25.850217: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0 2021-04-12 18:34:25.851868: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0 2021-04-12 18:34:25.856475: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2021-04-12 18:34:25.856617: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.857449: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.858136: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0 2021-04-12 18:34:25.858221: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2021-04-12 18:34:25.859743: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-04-12 18:34:25.859776: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2021-04-12 18:34:25.859792: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N 2021-04-12 18:34:25.859929: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.860759: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.861659: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7) Command: train Model: coco Dataset: /content/Mask_RCNN/dataset/coco/ Year: 2017 Logs: Mask_RCNNr/dataset/coco/logs Auto Download: False

Configurations: BACKBONE resnet101 BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] COMPUTE_BACKBONE_SHAPE None DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.7 DETECTION_NMS_THRESHOLD 0.3 FPN_CLASSIF_FC_LAYERS_SIZE 1024 GPU_COUNT 1 GRADIENT_CLIP_NORM 5.0 IMAGES_PER_GPU 1 IMAGE_CHANNEL_COUNT 3 IMAGE_MAX_DIM 1024 IMAGE_META_SIZE 93 IMAGE_MIN_DIM 800 IMAGE_MIN_SCALE 0 IMAGE_RESIZE_MODE square IMAGE_SHAPE [1024 1024 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME coco NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 PRE_NMS_LIMIT 6000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (32, 64, 128, 256, 512) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.5 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TOP_DOWN_PYRAMID_SIZE 256 TRAIN_BN False TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From /root/.local/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py:553: The name tf.random_shuffle is deprecated. Please use tf.random.shuffle instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/utils.py:202: The name tf.log is deprecated. Please use tf.math.log instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py:600: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead Loading weights /content/Mask_RCNN/pretrained_model/coco/mask_rcnn_coco.h5 2021-04-12 18:34:34.295531: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.296325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235 pciBusID: 0000:00:04.0 2021-04-12 18:34:34.296445: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2021-04-12 18:34:34.296496: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0 2021-04-12 18:34:34.296546: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0 2021-04-12 18:34:34.296595: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0 2021-04-12 18:34:34.296640: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0 2021-04-12 18:34:34.296688: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0 2021-04-12 18:34:34.296732: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2021-04-12 18:34:34.296833: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.297626: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.298317: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0 2021-04-12 18:34:34.298383: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-04-12 18:34:34.298404: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2021-04-12 18:34:34.298419: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N 2021-04-12 18:34:34.298540: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.299287: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.299997: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7) loading annotations into memory... Done (t=0.06s) creating index... index created! loading annotations into memory... Done (t=0.01s) creating index... index created! Training network heads

Starting at epoch 0. LR=0.001

Checkpoint Path: Mask_RCNNr/dataset/coco/logs/coco20210412T1834/mask_rcnncoco{epoch:04d}.h5 Selecting layers to train fpn_c5p5 (Conv2D) fpn_c4p4 (Conv2D) fpn_c3p3 (Conv2D) fpn_c2p2 (Conv2D) fpn_p5 (Conv2D) fpn_p2 (Conv2D) fpn_p3 (Conv2D) fpn_p4 (Conv2D) In model: rpn_model rpn_conv_shared (Conv2D) rpn_class_raw (Conv2D) rpn_bbox_pred (Conv2D) mrcnn_mask_conv1 (TimeDistributed) mrcnn_mask_bn1 (TimeDistributed) mrcnn_mask_conv2 (TimeDistributed) mrcnn_mask_bn2 (TimeDistributed) mrcnn_class_conv1 (TimeDistributed) mrcnn_class_bn1 (TimeDistributed) mrcnn_mask_conv3 (TimeDistributed) mrcnn_mask_bn3 (TimeDistributed) mrcnn_class_conv2 (TimeDistributed) mrcnn_class_bn2 (TimeDistributed) mrcnn_mask_conv4 (TimeDistributed) mrcnn_mask_bn4 (TimeDistributed) mrcnn_bbox_fc (TimeDistributed) mrcnn_mask_deconv (TimeDistributed) mrcnn_class_logits (TimeDistributed) mrcnn_mask (TimeDistributed) /root/.local/lib/python3.7/site-packages/tensorflow/python/ops/gradients_util.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " /root/.local/lib/python3.7/site-packages/tensorflow/python/ops/gradients_util.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " /root/.local/lib/python3.7/site-packages/tensorflow/python/ops/gradients_util.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

Epoch 1/20 sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 3772, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/3772.jpg', 'width': 827, 'height': 1169, 'annotations': [{'area': 83082, 'iscrowd': 0, 'image_id': 3772, 'bbox': [40, 548, 366, 227], 'category_id': 2, 'id': 12755, 'ignore': 0, 'segmentation': []}, {'area': 77827, 'iscrowd': 0, 'image_id': 3772, 'bbox': [406, 548, 349, 223], 'category_id': 2, 'id': 12756, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 5819, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/5819.jpg', 'width': 825, 'height': 1075, 'annotations': [{'area': 141255, 'iscrowd': 0, 'image_id': 5819, 'bbox': [84, 259, 657, 215], 'category_id': 1, 'id': 10957, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 7571, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/7571.jpg', 'width': 827, 'height': 1063, 'annotations': [{'area': 54496, 'iscrowd': 0, 'image_id': 7571, 'bbox': [235, 189, 208, 262], 'category_id': 3, 'id': 10756, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 8432, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/8432.jpg', 'width': 906, 'height': 1170, 'annotations': [{'area': 13824, 'iscrowd': 0, 'image_id': 8432, 'bbox': [127, 857, 216, 64], 'category_id': 4, 'id': 13861, 'ignore': 0, 'segmentation': []}, {'area': 168399, 'iscrowd': 0, 'image_id': 8432, 'bbox': [445, 236, 297, 567], 'category_id': 1, 'id': 13862, 'ignore': 0, 'segmentation': []}, {'area': 14307, 'iscrowd': 0, 'image_id': 8432, 'bbox': [449, 865, 251, 57], 'category_id': 4, 'id': 13863, 'ignore': 0, 'segmentation': []}, {'area': 6660, 'iscrowd': 0, 'image_id': 8432, 'bbox': [127, 1002, 74, 90], 'category_id': 4, 'id': 13864, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 4528, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/4528.jpg', 'width': 825, 'height': 1075, 'annotations': [{'area': 422676, 'iscrowd': 0, 'image_id': 4528, 'bbox': [58, 60, 708, 597], 'category_id': 1, 'id': 6253, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 1084, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/1084.jpg', 'width': 850, 'height': 1100, 'annotations': [{'area': 194208, 'iscrowd': 0, 'image_id': 1084, 'bbox': [18, 196, 816, 238], 'category_id': 1, 'id': 4828, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range Traceback (most recent call last): File "samples/coco/coco.py", line 576, in augmentation=augmentation) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 2375, in train File "/usr/local/lib/python3.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1732, in fit_generator initial_epoch=initial_epoch) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training_generator.py", line 185, in fit_generator generator_output = next(output_generator) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range

`

I think this caused by the lack of segmentation annotation, please anyone help me solve this problem ! Are there any tool that can create segmentation annotation from bbox? my dataset is large so that I dont want to re-annotate it 😭😭

thank you all !

sohinimallick commented 3 years ago

I have the same problem...my annotations are in the pascal voc format with only bounding box co-ordinates. Suggestions will be super helpful!

konstantin-frolov commented 3 years ago

1) You can create fully filled masks for using MRCNN. Add into json file in segmentation field [[[y0, x0], ... [yN,xN], [y0,x,0]]], where yi, xi - bbox coordinates. 2) use Faster RCNN (only bbox and class detector)

Nhatcao99 commented 3 years ago

Hi, thank for great work my data is a json file in COCO format but it does not include any segmentation annotation :

{"area": 10712, "iscrowd": 0, "image_id": 379, "bbox": [557, 747, 103, 104], "category_id": 4, "id": 3286, "ignore": 0, "segmentation": []}

I got this error when using my data : ` WARNING:root:Fail load requirements file, so using default ones. /usr/local/lib/python3.7/dist-packages/setuptools/dist.py:645: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead % (opt, underscore_opt)) /usr/local/lib/python3.7/dist-packages/setuptools/dist.py:645: UserWarning: Usage of dash-separated 'license-file' will not be supported in future versions. Please use the underscore name 'license_file' instead % (opt, underscore_opt)) /usr/local/lib/python3.7/dist-packages/setuptools/dist.py:645: UserWarning: Usage of dash-separated 'requirements-file' will not be supported in future versions. Please use the underscore name 'requirements_file' instead % (opt, underscore_opt)) running install running bdist_egg running egg_info writing mask_rcnn.egg-info/PKG-INFO writing dependency_links to mask_rcnn.egg-info/dependency_links.txt writing top-level names to mask_rcnn.egg-info/top_level.txt warning: Failed to find the configured license file 'LICENSE' reading manifest file 'mask_rcnn.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'README.md' warning: no files found matching 'LICENSE' writing manifest file 'mask_rcnn.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/visualize.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/config.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/model.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/save_result.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/init.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/utils.py -> build/bdist.linux-x86_64/egg/mrcnn copying build/lib/mrcnn/parallel_model.py -> build/bdist.linux-x86_64/egg/mrcnn byte-compiling build/bdist.linux-x86_64/egg/mrcnn/visualize.py to visualize.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/config.py to config.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/model.py to model.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/save_result.py to save_result.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/init.py to init.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/utils.py to utils.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/mrcnn/parallel_model.py to parallel_model.cpython-37.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying mask_rcnn.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying mask_rcnn.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying mask_rcnn.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying mask_rcnn.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating 'dist/mask_rcnn-2.1-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing mask_rcnn-2.1-py3.7.egg Removing /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg Copying mask_rcnn-2.1-py3.7.egg to /usr/local/lib/python3.7/dist-packages mask-rcnn 2.1 is already the active version in easy-install.pth

Installed /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg Processing dependencies for mask-rcnn==2.1 Finished processing dependencies for mask-rcnn==2.1 [ ] !python3 samples/coco/coco.py train --dataset=/content/Mask_RCNN/dataset/coco/ --model=coco /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /root/.local/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /root/.local/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) Using TensorFlow backend. WARNING:tensorflow:From samples/coco/coco.py:75: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From samples/coco/coco.py:77: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-04-12 18:34:25.759206: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2021-04-12 18:34:25.763629: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2299995000 Hz 2021-04-12 18:34:25.763909: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55bb3a2b9340 executing computations on platform Host. Devices: 2021-04-12 18:34:25.763948: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , 2021-04-12 18:34:25.766009: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1 2021-04-12 18:34:25.841117: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.842035: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55bb3a2b9500 executing computations on platform CUDA. Devices: 2021-04-12 18:34:25.842072: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): Tesla K80, Compute Capability 3.7 2021-04-12 18:34:25.842262: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.843002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235 pciBusID: 0000:00:04.0 2021-04-12 18:34:25.843364: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2021-04-12 18:34:25.844951: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0 2021-04-12 18:34:25.846810: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0 2021-04-12 18:34:25.847213: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0 2021-04-12 18:34:25.850217: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0 2021-04-12 18:34:25.851868: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0 2021-04-12 18:34:25.856475: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2021-04-12 18:34:25.856617: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.857449: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.858136: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0 2021-04-12 18:34:25.858221: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2021-04-12 18:34:25.859743: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-04-12 18:34:25.859776: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2021-04-12 18:34:25.859792: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N 2021-04-12 18:34:25.859929: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.860759: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:25.861659: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7) Command: train Model: coco Dataset: /content/Mask_RCNN/dataset/coco/ Year: 2017 Logs: Mask_RCNNr/dataset/coco/logs Auto Download: False

Configurations: BACKBONE resnet101 BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] COMPUTE_BACKBONE_SHAPE None DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.7 DETECTION_NMS_THRESHOLD 0.3 FPN_CLASSIF_FC_LAYERS_SIZE 1024 GPU_COUNT 1 GRADIENT_CLIP_NORM 5.0 IMAGES_PER_GPU 1 IMAGE_CHANNEL_COUNT 3 IMAGE_MAX_DIM 1024 IMAGE_META_SIZE 93 IMAGE_MIN_DIM 800 IMAGE_MIN_SCALE 0 IMAGE_RESIZE_MODE square IMAGE_SHAPE [1024 1024 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME coco NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 PRE_NMS_LIMIT 6000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (32, 64, 128, 256, 512) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.5 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TOP_DOWN_PYRAMID_SIZE 256 TRAIN_BN False TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From /root/.local/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py:553: The name tf.random_shuffle is deprecated. Please use tf.random.shuffle instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/utils.py:202: The name tf.log is deprecated. Please use tf.math.log instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py:600: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version. Instructions for updating: box_ind is deprecated, use box_indices instead Loading weights /content/Mask_RCNN/pretrained_model/coco/mask_rcnn_coco.h5 2021-04-12 18:34:34.295531: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.296325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235 pciBusID: 0000:00:04.0 2021-04-12 18:34:34.296445: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2021-04-12 18:34:34.296496: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0 2021-04-12 18:34:34.296546: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0 2021-04-12 18:34:34.296595: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0 2021-04-12 18:34:34.296640: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0 2021-04-12 18:34:34.296688: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0 2021-04-12 18:34:34.296732: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2021-04-12 18:34:34.296833: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.297626: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.298317: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0 2021-04-12 18:34:34.298383: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-04-12 18:34:34.298404: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2021-04-12 18:34:34.298419: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N 2021-04-12 18:34:34.298540: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.299287: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-04-12 18:34:34.299997: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7) loading annotations into memory... Done (t=0.06s) creating index... index created! loading annotations into memory... Done (t=0.01s) creating index... index created! Training network heads

Starting at epoch 0. LR=0.001

Checkpoint Path: Mask_RCNNr/dataset/coco/logs/coco20210412T1834/mask_rcnncoco{epoch:04d}.h5 Selecting layers to train fpn_c5p5 (Conv2D) fpn_c4p4 (Conv2D) fpn_c3p3 (Conv2D) fpn_c2p2 (Conv2D) fpn_p5 (Conv2D) fpn_p2 (Conv2D) fpn_p3 (Conv2D) fpn_p4 (Conv2D) In model: rpn_model rpn_conv_shared (Conv2D) rpn_class_raw (Conv2D) rpn_bbox_pred (Conv2D) mrcnn_mask_conv1 (TimeDistributed) mrcnn_mask_bn1 (TimeDistributed) mrcnn_mask_conv2 (TimeDistributed) mrcnn_mask_bn2 (TimeDistributed) mrcnn_class_conv1 (TimeDistributed) mrcnn_class_bn1 (TimeDistributed) mrcnn_mask_conv3 (TimeDistributed) mrcnn_mask_bn3 (TimeDistributed) mrcnn_class_conv2 (TimeDistributed) mrcnn_class_bn2 (TimeDistributed) mrcnn_mask_conv4 (TimeDistributed) mrcnn_mask_bn4 (TimeDistributed) mrcnn_bbox_fc (TimeDistributed) mrcnn_mask_deconv (TimeDistributed) mrcnn_class_logits (TimeDistributed) mrcnn_mask (TimeDistributed) /root/.local/lib/python3.7/site-packages/tensorflow/python/ops/gradients_util.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " /root/.local/lib/python3.7/site-packages/tensorflow/python/ops/gradients_util.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " /root/.local/lib/python3.7/site-packages/tensorflow/python/ops/gradients_util.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/callbacks/tensorboard_v1.py:200: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/callbacks/tensorboard_v1.py:203: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

Epoch 1/20 sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 3772, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/3772.jpg', 'width': 827, 'height': 1169, 'annotations': [{'area': 83082, 'iscrowd': 0, 'image_id': 3772, 'bbox': [40, 548, 366, 227], 'category_id': 2, 'id': 12755, 'ignore': 0, 'segmentation': []}, {'area': 77827, 'iscrowd': 0, 'image_id': 3772, 'bbox': [406, 548, 349, 223], 'category_id': 2, 'id': 12756, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 5819, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/5819.jpg', 'width': 825, 'height': 1075, 'annotations': [{'area': 141255, 'iscrowd': 0, 'image_id': 5819, 'bbox': [84, 259, 657, 215], 'category_id': 1, 'id': 10957, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 7571, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/7571.jpg', 'width': 827, 'height': 1063, 'annotations': [{'area': 54496, 'iscrowd': 0, 'image_id': 7571, 'bbox': [235, 189, 208, 262], 'category_id': 3, 'id': 10756, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 8432, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/8432.jpg', 'width': 906, 'height': 1170, 'annotations': [{'area': 13824, 'iscrowd': 0, 'image_id': 8432, 'bbox': [127, 857, 216, 64], 'category_id': 4, 'id': 13861, 'ignore': 0, 'segmentation': []}, {'area': 168399, 'iscrowd': 0, 'image_id': 8432, 'bbox': [445, 236, 297, 567], 'category_id': 1, 'id': 13862, 'ignore': 0, 'segmentation': []}, {'area': 14307, 'iscrowd': 0, 'image_id': 8432, 'bbox': [449, 865, 251, 57], 'category_id': 4, 'id': 13863, 'ignore': 0, 'segmentation': []}, {'area': 6660, 'iscrowd': 0, 'image_id': 8432, 'bbox': [127, 1002, 74, 90], 'category_id': 4, 'id': 13864, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 4528, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/4528.jpg', 'width': 825, 'height': 1075, 'annotations': [{'area': 422676, 'iscrowd': 0, 'image_id': 4528, 'bbox': [58, 60, 708, 597], 'category_id': 1, 'id': 6253, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range sdfsdfsdfsdf isinstance(segm, list) ERROR:root:Error processing image {'id': 1084, 'source': 'coco', 'path': '/content/Mask_RCNN/dataset/coco//train2017/1084.jpg', 'width': 850, 'height': 1100, 'annotations': [{'area': 194208, 'iscrowd': 0, 'image_id': 1084, 'bbox': [18, 196, 816, 238], 'category_id': 1, 'id': 4828, 'ignore': 0, 'segmentation': []}]} Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator use_mini_mask=config.USE_MINI_MASK) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt mask, class_ids = dataset.load_mask(image_id) File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range Traceback (most recent call last): File "samples/coco/coco.py", line 576, in augmentation=augmentation) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 2375, in train File "/usr/local/lib/python3.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1732, in fit_generator initial_epoch=initial_epoch) File "/usr/local/lib/python3.7/dist-packages/keras/engine/training_generator.py", line 185, in fit_generator generator_output = next(output_generator) File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1709, in data_generator File "/usr/local/lib/python3.7/dist-packages/mask_rcnn-2.1-py3.7.egg/mrcnn/model.py", line 1212, in load_image_gt File "samples/coco/coco.py", line 273, in load_mask image_info["width"]) File "samples/coco/coco.py", line 343, in annToMask rle = self.annToRLE(ann, height, width) File "samples/coco/coco.py", line 321, in annToRLE rles = maskUtils.frPyObjects(segm, height, width) File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects IndexError: list index out of range

`

I think this caused by the lack of segmentation annotation, please anyone help me solve this problem ! Are there any tool that can create segmentation annotation from bbox? my dataset is large so that I dont want to re-annotate it 😭😭

thank you all !

Is that you :v

Nhatcao99 commented 3 years ago
  1. You can create fully filled masks for using MRCNN. Add into json file in segmentation field [[[y0, x0], ... [yN,xN], [y0,x,0]]], where yi, xi - bbox coordinates.
  2. use Faster RCNN (only bbox and class detector)

Thank you very much for the answer. May I have some question.

Does y stand before x or x stand before y. Kind of odd seeing y stand before x?

I am trying to convert bounding box to mask as well. This is my first time using it. Can I as does these point need order or you can place them in any order? for example I have the all the point of the bounding box as follow: [x_min,y_min][x_max, y _mix][x_min y_max][x_max y_max] Do I have to put them into any special order? And also, is it right syntax to put them like this supposed y stand before x: segmentation: [[y_min, x_min],[],[],[y_max, x_max]]? Please help me!!! THank you

konstantin-frolov commented 3 years ago

Please help me!!! THank you

ok, just look into annotations json. In field annotations you have annotation id, image_id, height, width, is_crowd, area, bbox, segmentation Bbox coordinates - it is 1x4 vector [y,x,height,width] Segmentations - it is Mx2N vector, where M - numbers of objects in segmentation(if only one object in bbox M=1) and N - number of points. segmentation typical view for one object in bbox is [[y0,x0,y1,x1, ..., yN,xN]]. you need convert bbox from yxhw format to yxyx format. Than create segmentation vector as [[y0, x0, y1, x0, y1, x1, y0, x1, y0, x0]] for each annotation in json file. And change area as bbox_width * bbox_height. you can inspect your dataset in Inspect_data notebook. It is very helpful.

Nhatcao99 commented 3 years ago

Please help me!!! THank you

ok, just look into annotations json. In field annotations you have annotation id, image_id, height, width, is_crowd, area, bbox, segmentation Bbox coordinates - it is 1x4 vector [y,x,height,width] Segmentations - it is Mx2N vector, where M - numbers of objects in segmentation(if only one object in bbox M=1) and N - number of points. segmentation typical view for one object in bbox is [[y0,x0,y1,x1, ..., yN,xN]]. you need convert bbox from yxhw format to yxyx format. Than create segmentation vector as [[y0, x0, y1, x0, y1, x1, y0, x1, y0, x0]] for each annotation in json file. And change area as bbox_width * bbox_height. you can inspect your dataset in Inspect_data notebook. It is very helpful.

THank you so much