search

facebookresearch / CutLER

Code release for "Cut and Learn for Unsupervised Object Detection and Instance Segmentation" and "VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation"
Other
918 stars 90 forks source link

Pretraining with CutLER #15

Closed VGrondin closed 1 year ago

VGrondin commented 1 year ago

Hello, reading your paper was pretty interesting and insightful. I was wondering how well an object detector model such as ViTDet can benefit by pretraining with CutLER? For instance, from the ViTDet paper, the authors acheive 55.6 APbox and 49.2. APmask (table 5 in Exploring Plain Vision Transformer Backbones for Object Detection), so is it possible to pretrain a ViTDet with CutLER and finetune it in a supervised learning way on COCO to improve detection results?

Thanks again for the great paper.

frank-xwang commented 1 year ago

Hi @VGrondin, I believe that using ViTDet pretrained with CutLER could potentially improve performance for semi-/fully supervised learning. I would be interested to hear about any updates you may have on any observed performance gains. Thank you!

VGrondin commented 1 year ago

Hi @frank-xwang, thanks for the information. My interest in CutLER mostly lies in the performance improvement on a custom dataset of forest images. I will let you know how it goes, I prepared the pseudo masks using maskcut with a mae backbone pretrained on 100k forest images (example of pseudo mask generated): sample0 sample1

Now I am trying to load vitdet config in cutler, but I am having some compatibility issues between the .yaml config and the recent .py config that vitdet uses. I will let you know how it goes once I find a way to resolve this issue.