facebookresearch / nougat

Implementation of Nougat Neural Optical Understanding for Academic Documents
https://facebookresearch.github.io/nougat/
MIT License
8.98k stars 567 forks source link

Questions about PDF to LaTeX Conversion #69

Closed DevanshuBrahmbhatt closed 1 year ago

DevanshuBrahmbhatt commented 1 year ago

I've recently started exploring PDF to LaTeX conversion, and I have a few questions:

  1. Is the tool/repository you are discussing already trained on something specific? If so, could you provide some details on the training data or pre-trained models used?
  2. How well does it handle tables that span across two or more pages? Can it automatically split and format them correctly in LaTeX?
  3. What about complex merged tables with multiple columns and rows? How does the tool/repository handle such scenarios?

I'd appreciate any insights or recommendations you can provide. Thanks in advance for your help!

sondo01 commented 1 year ago

I have the same question. Additionally, can Nougat convert tables from images embedded in a PDF file? No?

marwinsteiner commented 1 year ago

I think reading the paper available on arxiv which you can get to via the link to the Facebook Research project page should answer most of these questions in sufficient detail.

  1. repo is trained on recognizing academic papers in the format found on Arxiv for the most part. Works very well with English and other Roman languages. Other languages currently not supported
  2. probably not well, again, your best bet of answering this question would be to read the paper by Lukas Blecher et al.
  3. see point 2 above

don't think nougat can handle images of tables to latex tables.

lukas-blecher commented 1 year ago

Thanks Marwin, that's correct.

Nougat can do many simple and even some more complex tables, but it struggles with page filling tables (or even larger than that)

For embedded tables, it could work. But it would have to look similar to regular latex tables. Don't know how well it works for spreadsheet for example