Poor mask prediction on image's boundary

[enchantress1016]

Hi, I tried different configs with the network and realized for a lot of time the mask prediction would lose the mask on the image boundary. My mask usually would be a rectangular shape lying on several rows, from very left to very right of the image, but the predicted masks sometime would either miss detect the very left or the very right parts, for several pixels. Any idea about why this would happen and how I could change the config to improve? Thanks!

FYI, all my training and testing data have the same size (1024, 1024, 3) (so I think probably I don't need to pad anymore) And I have the config as following: Configurations: BACKBONE resnet101 BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.98 DETECTION_NMS_THRESHOLD 0.3 GPU_COUNT 1 GRADIENT_CLIP_NORM 5.0 IMAGES_PER_GPU 1 IMAGE_MAX_DIM 1024 IMAGE_META_SIZE 14 IMAGE_MIN_DIM 1024 IMAGE_MIN_SCALE 0 IMAGE_RESIZE_MODE none IMAGE_SHAPE [1024 1024 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_bbox_loss': 1.0} MASK_POOL_SIZE 70 MASK_SHAPE [56, 56] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME Mtc NUM_CLASSES 2 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (32, 64, 128, 256, 512) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 512 STEPS_PER_EPOCH 100 TRAIN_BN False TRAIN_ROIS_PER_IMAGE 512 USE_MINI_MASK False USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001

[fastlater]

@enchantress1016 May I ask you what is the effect of MASK_POOL_SIZE = 70 and MASK_SHAPE = [56, 56]? The original script values are 14 and [28,28] respectively. Do they increase the accuracy in 1024*1024 images?

About your question, one suggestion: Did you try to check your resulted model with https://github.com/matterport/Mask_RCNN/blob/master/samples/nucleus/inspect_nucleus_model.ipynb

btw, dont you think your DETECTION_MIN_CONFIDENCE is too high? This mean that the model has to be 98% sure that x pixel belongs to y class so it may be causing the error in the border. It is just my opinion.

[enchantress1016]

@fastlater Hi, thank you for your suggestion! I'll read through the link you posted, hope it can inspire me :)

Regarding of mask_shape [56, 56] and mask_pool_size 70, I adjusted it to increase the resolution of my mask. I have some small variations on the mask edge but the original config was not able to detect it. After making these changes, you can see a clear resolution increase on your detected masks, although I need it to be even better ;)

[fastlater]

@enchantress1016
Get good results is easy with this state of art script. However, reach a really high accuracy will need some extra work. May some preprocessing or postprocessing. I read this post from the kaggle's dsb forum https://www.kaggle.com/c/data-science-bowl-2018/discussion/54741 and helped me a lot. Actually, this competition's forum is full of discussions with good training tips. I still don't reach the desired accuracy either but I will try changing those mask shape and mask pool size and see what happen since my input images have the same size than yours and the rest of my parameters are almost the same than yours. Did you used imagenet, coco or trained from scratch? For me, COCO is slightly better than Imagenet but testing is the only way to know which one fits better your data.

[enchantress1016]

@fastlater I used coco as my starting point, for my dataset Imagenet performs much worse than coco but I didn't figure out the reason for it. Remember when you change the mask shape, you also need to change the architecture by adding an additional deconv layer in model.py. I did see a clear performance improvement with higher mask_pool_size value when do the testing. However, when training the network the max mask_pool_size I could use is 28 (otherwise there'd be OOM error). So I trained on mask_pool_size 28 and predicted on mask_pool_size 70, maybe you can try increasing the mask_pool_size in training if your hardware allows.

[fastlater]

I see, about the weights, well, some cases Imagenet is better, other cases COCO is better so there is nothing strange with that. You can try using a different optimizer, like ADAM for example. Some research papers used ADAM as optimizer. But is it just like the starting weights issue, depends on case.

[enchantress1016]

@fastlater That's a really good point. I thought the code used ADAM by default so I never thought about changing the optimizer. Also, I tried to use weighted DICE+BCE as loss function, but didn't see clear improvement for my data. Maybe you can try it on yours, as some people indeed got better boundary detected.

[florian-koenig]

@fastlater That's a really good point. I thought the code used ADAM by default so I never thought about changing the optimizer. Also, I tried to use weighted DICE+BCE as loss function, but didn't see clear improvement for my data. Maybe you can try it on yours, as some people indeed got better boundary detected.

@enchantress1016 Can you please point me to a description of how to use DICE+BCE? I'd like to try if it produces better boundary in my use case as well. Thanks!

[K-M-Ibrahim-Khalilullah]

@waleedka How can I improve predicted Mask. Because My detected bounding box is good but Mask is not good. How many ways to improve Mask?

Thanks

[banafsh89]

@waleedka How can I improve predicted Mask. Because My detected bounding box is good but Mask is not good. How many ways to improve Mask?

Thanks

@ibrahimLearning were you able to improve your predicted mask? I have the same problem.