search

amazon-science / semimtr-text-recognition

Multimodal Semi-Supervised Learning for Text Recognition (SemiMTR)
Apache License 2.0
81 stars 12 forks source link

semimtr_finetune without language model use #12

Closed WongVi closed 1 year ago

WongVi commented 1 year ago

is it possible to fineture semimtr without trained language model use? if possible please let me know how? Thank you

aaberdam commented 1 year ago

Please elaborate more on why you don't want to use the language model. Is it due to its latency, or since you want to avoid pretraining of the language model. If you want to reduce latency, you can work only with the vision model predictions (you can take a look here: configs/semimtr_pretrain_vision_model.yaml). If you want to skip the pretraining of the language model, you can simply skip this stage and fine-tune the entire model, while the language model is initialized from scratch. This will decrease performance; however, if your train set is large enough, it won't be that significant.

WongVi commented 1 year ago

@aaberdam

I tried to run demo.py with vision pretrained model only but got error image

could you please let me know how can I solve?