search

kermitt2 / grobid

A machine learning software for extracting information from scholarly documents
https://grobid.readthedocs.io
Apache License 2.0
3.47k stars 447 forks source link

Not extracting when PDF is large #1162

Open victorcasignia opened 1 month ago

victorcasignia commented 1 month ago

Using Docker to run the service.

Used on a 21 pages PDF. It only extracts up to page 9 then it jumps to the bibliography. How do I resolve this?

image

image

lfoppiano commented 1 month ago

Hi @victorcasignia, which document are you processing? is it a scientific article?

Could you provide the document so I can make some tests?

victorcasignia commented 3 weeks ago

@lfoppiano Yes. I used this document https://arxiv.org/pdf/2307.01952

lfoppiano commented 3 weeks ago

Hi @victorcasignia I did went through the document and the body seems to be correctly processed. You can double check that the body is all in the output. Even the head of sections are numbered correctly.

Now, the issues are all in the Appendix, which is larger than the body of the article. The first part are correctly handlded until "blue" (where the document ends). After that the model decided that it's body so all the content after page 21 is actually appended after the "Future work" section.

From our end, we could flag this issue so that we can use the document as training data for the fulltext model, but will be for version 0.8.2.