search

NormXU / nougat-latex-ocr

Codebase for fine-tuning / evaluating nougat-based image2latex generation models
https://arxiv.org/abs/2308.13418
Apache License 2.0
115 stars 13 forks source link

Is it specifically trained for the latex formula? #1

Closed smclh closed 11 months ago

smclh commented 11 months ago

Hello, is this code specifically designed for training the latex formula? If it is text+formula, will it no longer work

NormXU commented 11 months ago

@smclh No, I think it should work.

You can utilize the codes to fine-tune for various image-to-text tasks. The text associated with each image does not have to be in LaTeX code format; any text format will be fine. However, it may be necessary to customize the dataset to align with the specific textual content you intend to train for. Like the code here.

Besides, since the codes are to fine-tune from the pre-trained nougat models, if your target text format is part of nougat-pretraining data, to recover the markdown format of arxiv paper, I believe the fine-tuning will be quite effective.