Mask-RCNN - predict not labeled picture - convert to coco format

[yc0619]

Hi,

related to the issue #4768

what i want to do is:

1. predict with not labeled pic
2. get segm as results
3. convert the saved segm result to coco format
4. convert coco format to labelme format, so that i can fix the wrong predicted label

i have tried your command to get segm_result and bbox_result from mask_rcnn. ./tools/dist_test.sh configs/_cfv/mask_rcnn_r50_fpn_2x_coco.py work_dir/r50best.pth 1 --format-only --options "jsonfile_prefix=./mask_rcnn_test-dev_results"

and i think the test is running, but at the end it comes, and the semg and bbox results are empty.

subprocess.CalledProcessError: Command '['/home/chao/anaconda3/envs/mmlab/bin/python', '-u', './tools/test.py', '--local_rank=0', 'configs/_cfv/mask_rcnn_r50_fpn_2x_coco.py', 'work_dir/r50best.pth', '--launcher', 'pytorch', '--format-only', '--options', 'jsonfile_prefix=./mask_rcnn_test-dev_results']' returned non-zero exit status 1.

What i've made before run the cmd:

1. label the predict file (if not labeled, no ann.json, comes an error, so i have just labeled one GT on the pic)
2. convert it to coco format

I would like to ask, if i can use test.py to predict the no labeled images. And how can i do it?

[hhaAndroid]

Thank you for your feedback. I think this feature is very important, but it is not currently supported. We will support no gt prediction mode ASAP.

[yc0619]

ok, is it possible, to test with a not full labeled pic? And get the full segm result?

[yc0619]

And how long might take the new version that support prediction with no labeled images? Sorry for asking that, i am making a project with a deadline. So ...

[yc0619]

[fake prediction - per test]

1. use labelme to produce at least 1 lbl/img
2. convert the labelme.json to coco format
3. save the imgs in ./data/ and save the ann.json at ./annotation.json
4. run the code to get segm.json ./tools/dist_test.sh configs/models/mask_rcnn_r50_fpn_2x_coco.py work_dirs/saved.pth 1 --format-only --options "jsonfile_prefix=./mask_rcnn_test-dev_results"

I am still working on how to convert the segm.json from mmdetection result to labelme. It isn't not a beautiful way to do the prediction. But it works at least. Hope it could help someone, who need the prediction function.

[hhaAndroid]

Sorry! No specific time has been set for now, but if you are interested, you can create a PR.

[yc0619]

Sure, no problem. I will try my best. Haha! Thanks again!

[hhaAndroid]

Looking forward to your contribution！

[Tarazed]

Hello, I'm also confused about predicting without label. After changing the config _base_/datasets/coco_detection.py line 45 to ann_file=None and line 46 to img_prefix=data_root + 'test2017/', it caused an error: ` tools/test.py:96: UserWarning: --options is deprecated in favor of --eval-options warnings.warn('--options is deprecated in favor of --eval-options') Traceback (most recent call last): File "/root/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 179, in build_from_cfg return obj_cls(**args) File "/root/mmdetection/mmdet/datasets/custom.py", line 87, in init self.data_infos = self.load_annotations(self.ann_file) File "/root/mmdetection/mmdet/datasets/coco.py", line 58, in load_annotations self.cat_ids = self.coco.get_cat_ids(cat_names=self.CLASSES) File "/root/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/pycocotools/coco.py", line 206, in get_cat_ids return self.getCatIds(cat_names, sup_names, cat_ids) File "/root/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/pycocotools/coco.py", line 192, in getCatIds cats = self.dataset['categories'] KeyError: 'categories'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "tools/test.py", line 220, in main() File "tools/test.py", line 163, in main dataset = build_dataset(cfg.data.test) File "/root/mmdetection/mmdet/datasets/builder.py", line 71, in build_dataset dataset = build_from_cfg(cfg, DATASETS, default_args) File "/root/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 182, in build_from_cfg raise type(e)(f'{obj_cls.name}: {e}') KeyError: "CocoDataset: 'categories'" `

Is this the correct way to predict without label?

[yc0619]

Hi,

i have implemented on my own dataset. I think you can maybe use pytorch to load your trained model and write a predict line.

I have tried it on mmdetection as the same way as the testpipeline. Before the prediction, i have added on my data some fake labels and convert them to coco-format. Then i run the testpipeline to get the true predicted segmentation and bbox.

[Tarazed]

Hi,

i have implemented on my own dataset. I think you can maybe use pytorch to load your trained model and write a predict line.

I have tried it on mmdetection as the same way as the testpipeline. Before the prediction, i have added on my data some fake labels and convert them to coco-format. Then i run the testpipeline to get the true predicted segmentation and bbox.

Thank you for your reply. It's an effective way to generate fake labels and then predict. I will try it later.

[pcicales]

@hhaAndroid this would be a very useful feature for high throughput prediction generation. Creating the dummy json files is also a viable solution, but more time consuming.

[yc0619]

@hhaAndroid this would be a very useful feature for high throughput label generation. Creating the dummy json files is also a viable solution, but more time consuming.

I use labelme to annotate my data. So i have set a dict as the labelme.json and give every picture, which needs to be predicted an annotation at the middle of the picture in the same size and a class for it. Then convert it in batch as coco-format dataset. It's quite easy by making dummy json file. 👍 Good luck by implement!

[pcicales]

@yc0619 Thank you! Yes it isn't bad to make the annotations, I did it similarly using a simple python script. The problem is when we want to run test on very large sets of data (100k+ or 1M+), where loading the dummy annotations can slow eval times.