How to improve accuracy (Model on TrOCR)

[C108152117]

I use TrOCR model to generate the math expression LATEX sequence according to the handwritten math expression image . but the consequences are unsatisfactory ;below are eval/loss 、train loss and the repository :https://github.com/win5923/TrOCR-Handwritten-Mathematical-Expression-Recognition

I use CROHME 2014 to test this model but worse than above image's models

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[ekdma7077]

Not exactly, but i think your pretrained_model's tokenizer is not optimized your math latex. so

you should use trained latex tokenizer. find it or collect latex data and create a new tokenizer using sentencepiece

[BizmanSS]

does training a new tokenizer not mess with the pretrained weights? since they use a specific tokenizer.