facebookresearch / nougat

Implementation of Nougat Neural Optical Understanding for Academic Documents
https://facebookresearch.github.io/nougat/
MIT License
8.98k stars 567 forks source link

nougat got stuck for testing. #86

Closed hongyi-zhao closed 1 year ago

hongyi-zhao commented 1 year ago

On Ubuntu, I try to do the testing with nougat, but got stuck as follows:

werner@X10DAi:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:    22.04
Codename:   jammy

$ pyenv shell datasci
$ pip install nougat-ocr[api,dataset]
$ proxychains-ng-socks5-country-control nougat Algorithms-for-Crystallographic-Groups.pdf -o .
WARNING:root:No GPU found. Conversion on CPU is very slow.
downloading nougat checkpoint version 0.1.0-small to path /home/werner/.cache/torch/hub/nougat-0.1.0-small
config.json: 100%|██████████████████████████████████████████████████████████████████| 557/557 [00:00<00:00, 2.73Mb/s]
pytorch_model.bin: 100%|██████████████████████████████████████████████████████████| 956M/956M [01:03<00:00, 15.7Mb/s]
special_tokens_map.json: 100%|█████████████████████████████████████████████████████| 96.0/96.0 [00:00<00:00, 249kb/s]
tokenizer.json: 100%|███████████████████████████████████████████████████████████| 2.04M/2.04M [00:01<00:00, 2.02Mb/s]
tokenizer_config.json: 100%|█████████████████████████████████████████████████████████| 106/106 [00:00<00:00, 552kb/s]
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
  0%|                                                                                         | 0/28 [00:00<?, ?it/s]INFO:root:Processing file Algorithms-for-Crystallographic-Groups.pdf with 28 pages
  4%|██▊                                                                           | 1/28 [03:14<1:27:37, 194.71s/it]

Any tips for this issue?

Regards, Zhao

evelynmitchell commented 1 year ago

The code is very slow without a GPU.

hongyi-zhao commented 1 year ago

Thank you for the tip. After installing cuda, its working performance is basically acceptable, as shown below:

werner@X10DAi:~$ sudo nvidia-smi 
Sun Sep 17 17:01:11 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 2070 ...    On  | 00000000:02:00.0  On |                  N/A |
|  0%   48C    P2              51W / 215W |    290MiB /  8192MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      2602      G   /usr/lib/xorg/Xorg                           64MiB |
|    0   N/A  N/A     14282    C+G   ...libexec/gnome-remote-desktop-daemon      102MiB |
|    0   N/A  N/A     14322      G   /usr/bin/gnome-shell                        117MiB |
+---------------------------------------------------------------------------------------+

(datasci) werner@X10DAi:~/Desktop/works/ActaCrystA/orbnpg/references$ proxychains-ng-socks5-country-control nougat Algorithms-for-Crystallographic-Groups.pdf -o .
/home/werner/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
  0%|                                                                                                                                           | 0/14 [00:00<?, ?it/s][nltk_data] Downloading package words to /home/werner/nltk_data...
[nltk_data]   Unzipping corpora/words.zip.
INFO:root:Processing file Algorithms-for-Crystallographic-Groups.pdf with 28 pages
  7%|█████████▎                                                                                                                         | 1/14 [00:23<05:00, 23.09s/it]WARNING:root:Found repetitions in sample 1
WARNING:root:Skipping page 4 due to repetitions.
 29%|█████████████████████████████████████▍                                                                                             | 4/14 [01:22<03:22, 20.21s/it]WARNING:root:Found repetitions in sample 1
WARNING:root:Skipping page 10 due to repetitions.
 79%|██████████████████████████████████████████████████████████████████████████████████████████████████████▏                           | 11/14 [03:22<00:52, 17.50s/it]WARNING:root:Found repetitions in sample 0
WARNING:root:Found repetitions in sample 1
WARNING:root:Skipping page 24 due to repetitions.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 14/14 [04:03<00:00, 17.41s/it]