Open LKWaters opened 1 year ago
Seems like we need to carry out a validity check before passing the polygons to TextRecInferencer. Thanks for reporting
I was able to solve this inelegantly by modifying base_mmocr_inferencer.py but hoping someone with more knowledge could find a better solution.
My changes to base_mmocr_inferencer.py From:
98 pipe_out = self.pipeline(inputs_)
To:
98 if not inputs_.size == 0: 99 pipe_out = self.pipeline(inputs_)
I changed the original code to your code but got the error: AttributeError: 'str' object has no attribute 'size' Then change it as follows:
98 if isinstance(inputs_, np.ndarray) and inputs_.size == 0: 99 pipe_out = self.pipeline(inputs_) 100 if pipe_out['data_samples'].get('img_path') is None: 101 pipe_out['data_samples'].set_metainfo( 102 dict(img_path=f'{self.num_unnamed_imgs}.jpg')) 103 self.num_unnamed_imgs += 1 104 chunk_data.append((inputs_, pipe_out))
Still got error:
Traceback (most recent call last): File "\\wsl$\Ubuntu-20.04\home\test\workspace\mmocr\tools\infer.py", line 100, in <module> main() File "\\wsl$\Ubuntu-20.04\home\test\workspace\mmocr\tools\infer.py", line 96, in main ocr(**call_args) File "\\wsl$\ubuntu-20.04\home\test\workspace\mmocr\mmocr\apis\inferencers\mmocr_inferencer.py", line 317, in __call__ preds = self.forward( File "\\wsl$\ubuntu-20.04\home\test\workspace\mmocr\mmocr\apis\inferencers\mmocr_inferencer.py", line 154, in forward result['det'] = self.textdet_inferencer( File "\\wsl$\ubuntu-20.04\home\test\workspace\mmocr\mmocr\apis\inferencers\base_mmocr_inferencer.py", line 196, in __call__ for ori_inputs, data in track( File "C:\Users\test\scoop\apps\python310\current\lib\site-packages\rich\progress.py", line 168, in track yield from progress.track( File "C:\Users\test\scoop\apps\python310\current\lib\site-packages\rich\progress.py", line 1210, in track for value in sequence: File "\\wsl$\ubuntu-20.04\home\test\workspace\mmocr\mmocr\apis\inferencers\base_mmocr_inferencer.py", line 80, in preprocess yield from map(self.collate_fn, chunked_data) File "C:\Users\test\scoop\apps\python310\current\lib\site-packages\mmengine\dataset\utils.py", line 64, in pseudo_collate data_item = data_batch[0] IndexError: list index out of range
Any suggestion to fix this bug? Thanks.
I tried my old fix and got the same results you did. Another very inelegant solution is to send a single white square when inputs_ is empty.
98 try:
99 pipe_out = self.pipeline(inputs_)
100 except:
101 pipe_out = self.pipeline(np.zeros([100,100,3],dtype=np.uint8)
This is again just a work around solution, Ill leave the proper solution to someone that knows mmocr better.
Prerequisite
Task
I'm using the official example scripts/configs for the officially supported tasks/models/datasets.
Branch
main branch https://github.com/open-mmlab/mmocr
Environment
sys.platform: linux Python: 3.8.16 (default, Jan 17 2023, 23:13:24) [GCC 11.2.0] CUDA available: True numpy_random_seed: 2147483648 GPU 0: Tesla T4 CUDA_HOME: /usr/local/cuda NVCC: Cuda compilation tools, release 11.7, V11.7.99 GCC: gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 PyTorch: 1.13.1+cu117 PyTorch compiling details: PyTorch built with:
TorchVision: 0.14.1+cu117 OpenCV: 4.5.3 MMEngine: 0.7.2 MMOCR: 1.0.0+d7c59f3
Reproduces the problem - code sample
mmocr_script.py
from mmocr.apis import MMOCRInferencer rec = 'abinet-vision_20e_st-an_mj' det = 'mask-rcnn_resnet50-oclip_fpn_160e_icdar2015' model = MMOCRInferencer(det=self.det, rec=self.rec)
result = model('path/to/local/image.jpg')
Reproduces the problem - command or script
python mmocr_script.py
Reproduces the problem - error message
[2023-04-30 06:39:54,775: ERROR/ForkPoolWorker-1] Task mmocr.ocr.detect[277fe791-f0ba-4e31-8f0c-a6f18759870b] raised unexpected: Exception() Traceback (most recent call last): File "/home/user/mmocr/ocr.py", line 135, in detect result = pipe.read_img(validation, go_words) File "/home/user/mmocr/ocr.py", line 36, in read_img result = model('image.jpg') File "/home/user/mmocr/mmocr/apis/inferencers/mmocr_inferencer.py", line 321, in call preds = self.forward( File "/home/user/mmocr/mmocr/apis/inferencers/mmocr_inferencer.py", line 175, in forward self.textrec_inferencer( File "/home/user/mmocr/mmocr/apis/inferencers/base_mmocr_inferencer.py", line 203, in call for ori_inputs, data in track( File "/home/user/miniconda3/lib/python3.8/site-packages/rich/progress.py", line 168, in track yield from progress.track( File "/home/user/miniconda3/lib/python3.8/site-packages/rich/progress.py", line 1210, in track for value in sequence: File "/home/user/mmocr/mmocr/apis/inferencers/base_mmocr_inferencer.py", line 80, in preprocess yield from map(self.collate_fn, chunked_data) File "/home/user/mmocr/mmocr/apis/inferencers/base_mmocr_inferencer.py", line 99, in _get_chunk_data pipeout = self.pipeline(inputs) File "/home/user/miniconda3/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 59, in call data = t(data) File "/home/user/miniconda3/lib/python3.8/site-packages/mmcv/transforms/base.py", line 12, in call return self.transform(results) File "/home/user/mmocr/mmocr/datasets/transforms/ocr_transforms.py", line 605, in transform results = super().transform(results) File "/home/user/miniconda3/lib/python3.8/site-packages/mmcv/transforms/processing.py", line 257, in transform self._resize_img(results) File "/home/user/mmocr/mmocr/datasets/transforms/ocr_transforms.py", line 554, in _resize_img return super()._resize_img(results) File "/home/user/miniconda3/lib/python3.8/site-packages/mmcv/transforms/processing.py", line 185, in _resize_img img, w_scale, h_scale = mmcv.imresize( File "/home/user/miniconda3/lib/python3.8/site-packages/mmcv/image/geometric.py", line 116, in imresize resized_img = cv2.resize( cv2.error: OpenCV(4.5.3) /tmp/pip-req-build-f51eratu/opencv/modules/imgproc/src/resize.cpp:4051: error: (-215:Assertion failed) !ssize.empty() in function 'resize'
Additional information
The exact image I'm using that fails: Image
The problem occurs because the detection portion of mmocr_inferencer.py is returning polygons that result in an array of shape (0,3,3). So there needs to be a way for detection to not return empty arrays.
I was able to solve this inelegantly by modifying base_mmocr_inferencer.py but hoping someone with more knowledge could find a better solution.
My changes to base_mmocr_inferencer.py From:
98 pipe_out = self.pipeline(inputs_)
To: