GPU outmemery - Githubissues

when run by GPU,errinfo: Loaded detection model vikp/surya_det3 on device cuda with dtype torch.float16 Loaded recognition model vikp/surya_rec2 on device cuda with dtype torch.float16 Detecting bboxes: 0%| | 0/6 [00:04<?, ?it/s] Traceback (most recent call last): File "/root/miniconda3/envs/surya/bin/surya_ocr", line 8, in sys.exit(main()) ^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/ocr_text.py", line 63, in main predictions_by_image = run_ocr(images, image_langs, det_model, det_processor, rec_model, rec_processor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/surya/ocr.py", line 65, in run_ocr det_predictions = batch_text_detection(images, det_model, det_processor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/surya/detection.py", line 128, in batch_text_detection preds, orig_sizes = batch_detection(images, model, processor, batch_size=batch_size) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/surya/detection.py", line 74, in batch_detection pred = model(pixel_values=batch) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/surya/model/detection/model.py", line 758, in forward logits = self.decode_head(encoder_hidden_states) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/surya/model/detection/model.py", line 729, in forward hidden_states = self.linear_fuse(torch.cat(all_hidden_states[::-1], dim=1)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 458, in forward return self._conv_forward(input, self.weight, self.bias) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/surya/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 454, in _conv_forward return F.conv2d(input, weight, bias, self.stride, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.09 GiB. GPU 0 has a total capacity of 9.77 GiB of which 460.81 MiB is free. Including non-PyTorch memory, this process has 9.31 GiB memory in use. Of the allocated memory 8.15 GiB is allocated by PyTorch, and 9 16.32 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#envi ronment-variables)

+-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found |

VikParuchuri / surya

GPU outmemery #202