WARNING:root:Skipping page 2 due to repetitions.

[darijgr]

Hi! I'm new to LLMs and coming at this from a user perspective. I've installed nougat on my Ubuntu WSL (Python 3.10.12) without any problems, but it behaves rather strangely on my input (a pretty good scan of a mathematical paper from 1979, attached to this ticket).

darij@skogen:/mnt/c/worksnap/temprepth/2$ nougat scan.pdf -o towber
WARNING:root:No GPU found. Conversion on CPU is very slow.
/home/darij/.local/lib/python3.10/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3526.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
  0%|                                                                                            | 0/49 [00:00<?, ?it/s]
INFO:root:Processing file scan.pdf with 49 pages
  2%|█▋                                                                               | 1/49 [03:07<2:30:00, 187.51s/it]WARNING:root:Found repetitions in sample 0
WARNING:root:Skipping page 2 due to repetitions.
  6%|████▉                                                                            | 3/49 [08:24<2:06:17, 164.74s/it]WARNING:root:Found repetitions in sample 0
WARNING:root:Skipping page 4 due to repetitions.
 12%|█████████▉                                                                       | 6/49 [17:14<2:06:29, 176.50s/it]
[... many lines omitted ...]
WARNING:root:Found repetitions in sample 0
100%|████████████████████████████████████████████████████████████████████████████████| 49/49 [2:08:21<00:00, 157.17s/it]
-> Cannot close object, library is destroyed. This may cause a memory leak!
-> Cannot close object, library is destroyed. This may cause a memory leak!

I'm pretty sure I have a GPU, so I don't know why it cannot find it. I also don't see where the possible memory leak at the end could be coming from. But most importantly, I don't understand why it is skipping pages and what these "repetitions" are. The output file is pretty good at the pages that it did not skip, but is missing the skipped ones. What's going on?

[darijgr]

scan.pdf Oh, and here is the scan.

[darijgr]

By the way, I'm getting the same problem (skipping page due to repetitions) on a different scan from another source. So it's not the PDF's fault probably.