frank-xwang
commented
1 year ago Hello @yeyan00, thank you for your interest in our work. However, please note that this repository is primarily intended for reproducing the results and experiments presented in our paper, and we may not be able to assist with debugging custom datasets. Nonetheless, we do have a few suggestions that could potentially improve your performance:
1) When conducting few-shot learning experiments, it is essential to tune the hyperparameters such as learning rates, batch size, and multiplier to achieve the best performance. Simply running the model for more epochs may not necessarily lead to better results, as overfitting can easily occur when only a limited number of samples are available. 2) You may want to consider using the mmdet's config and fine-tuning the model that has been pre-trained with CutLER. This could potentially improve the performance of the model on your custom dataset. It seems like that mmdet's config is working well. 3) It's worth noting that mmdet and detectron2 are two different codebases, so comparing the performance between them may not be entirely reliable. Several factors, such as learning rate, batch size, image size, augmentations, test settings (using multi-scale or not), can all affect the final performance. To ensure a fair comparison, I would suggest using the same codebase and settings to evaluate the performance. 4) I noticed that in your current configuration, you reduce the learning rate by a factor of 8, but the batch size is not similarly reduced. Typically, the two should be reduced by the same factor in most cases. However, it's essential to play with the hyperparameters and set them to different values to achieve the best results.
Hope these comments help. Xudong
yeyan00
1.train data use my dataset, about 6k images,size is 1024*1024, only box label, no mask label; config like coco-semisupervised, single-gpu, modify config: IMS_PER_BATCH=8; base_lr:0.04/8, max_iter:20000; about 30 epoch; MASK_ON:false INPUT: max_size_train:1024; min_size_train:(1024,); TEST: MAX_SIZE:1024,MIN_SIZES:1024
use prtrained model http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_cascade_final.pth
results of box AP: Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.264 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.421 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.280 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.081 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.217 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.366 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.291 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.438 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.441 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.147 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.366 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.523
2.when i use mmdet, mask-rcnn,use byol self self-supervised pretrained model, just train 12 epoch, the result is more better: Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.553 Average Precision (AP) @[ IoU=0.50 | area= all | =1000 ] = 0.831 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.632 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.218 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.507 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.610 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.634 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.634 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.634 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.238 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.570 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.688
(1) how can i improve the mAP? if config file is not right? (2) if i want to self-training use my dataset(unlabeled data is 60K ) ,should i do like this: step 1.use mast_cut.py and merge_json.py generate pseudo mask step 2.use pseudo mask, unsupervised Model Learning setp 3:self training
my dataset has some features: small object; complex scene;