matterport / Mask_RCNN

Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow
Other
24.65k stars 11.7k forks source link

Training from scratch (Without using pre-trained weights) #1399

Open Carlogo815 opened 5 years ago

Carlogo815 commented 5 years ago

Hi, has anyone tried to train the network from scratch? (Without using any of the pre-trained weights provided)

I am trying to do in and, in order to do so, I commented the lines where the weights are loaded. However, I am always getting AP = 0 and AR = 0 for both bounding boxes and masks. Is there something I am doing wrong? and which training schedule would be suitable to do it?

Here are the results I have obtained so far

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000 Prediction time: 110.37733197212219. Average 0.2207546639442444/image Total time: 118.04929542541504 Running COCO evaluation on 500 images for annotation type segm... Loading and preparing results... DONE (t=0.00s) creating index... index created! Running per image evaluation... Evaluate annotation type segm DONE (t=0.96s). Accumulating evaluation results... DONE (t=0.34s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000 Prediction time: 103.53374075889587. Average 0.20706748151779175/image Total time: 110.01924180984497

white2018 commented 5 years ago

I tried training from scratch using coco2014 dataset for one day. However, the loss remain around 3 all the time. don't know what's wrong with my training process

koukouzasg commented 5 years ago

Training from scratch takes many days so its normal if your loss did not decrease after only 1 day. For example, imagenet was trained for 2 weeks in 1.2 million images.

ChangyuanZhou commented 5 years ago

I encountered the same problem several days ago, and after checking the code I found the problems lied in that in the evaluation, the line loading the weights should not be commented. This is because in evaluation period, the main code should find and load the last trained model.

koukouzasg commented 5 years ago

Can you edit your comment providing the code you modified in order to help other people that might encounter the same problem?

Amrimn commented 4 years ago

@ChangyuanZhou @Carlogo815 can you share what modification you did please? Thanks

Pabligme commented 2 years ago

@Carlogo815 I answer very late, but I solved this problem change MEAN_PIXEL to [0,0,0] (or low values). I don't sure if this is correct. Also tried change MEAN_PIXEL to mean of pixel in each band RGB of my training dataset but this didn't work so well. However, I read that the mean of the RGB band of the training dataset is not always the best.

Any correction or contribution is welcome.