Does mmselfsup can be used to self supervised images of text to then evaluate on mmocr downstreams tasks?

open-mmlab / mmselfsup

OpenMMLab Self-Supervised Learning Toolbox and Benchmark

https://mmselfsup.readthedocs.io/en/latest/

Apache License 2.0

3.18k stars 429 forks source link

Does mmselfsup can be used to self supervised images of text to then evaluate on mmocr downstreams tasks? #292

Closed ykacer closed 2 years ago

ykacer commented 2 years ago

i'm working on mmocr but I'm very new to mmselfsup. Is it possible to use mmselfsup with my images of text and then use the backbone to train a mmocr model? Thanks

fangyixiao18 commented 2 years ago

Yes, it is possible to do so. However, currently the experiments of self supervised learning pretraining are done on ImageNet, which is a object-centric dataset and is different with images of text, so I guess dense contrastive learning methods (DenseCL) or masked image modeling methods (MAE, SimMIM) are more appropriate.