search

open-mmlab / mmdeploy

OpenMMLab Model Deployment Framework
https://mmdeploy.readthedocs.io/en/latest/
Apache License 2.0
2.69k stars 617 forks source link

[Bug] KeyError: 'img_path' #1716

Open allankouidri opened 1 year ago

allankouidri commented 1 year ago

Checklist

Describe the bug

The bug occurs when preparing the model inputs for backend model inference from a end2end model (e.g. dbnet_r50 or satrn) using np.ndarray as input.

There is no issue when using the image path.

By editing the mmocr lib, replacing the transform function from "class LoadImageFromFile(MMCV_LoadImageFromFile)" the one from "class LoadImageFromNDArray(LoadImageFromFile)" the inference is working.

Reproduction

import torch
from mmdeploy.apis.utils import build_task_processor
from mmdeploy.utils import get_input_shape, load_config
from PIL import Image
import numpy as np

model_cfg = 'mmocr/configs/textrecog/satrn/satrn_shallow-small_5e_st_mj.py'
deploy_cfg = 'mmdeploy/configs/mmocr/text-recognition/text-recognition_onnxruntime_dynamic.py'
backend_files = ['work_dir/onnx/satrn_shallow-small_finetuned/end2end.onnx']
img_path = "C:/Users/poopo/Desktop/Ikomia/Images/demo_text_recog.jpg"
device = "cpu"

# load the image
image = Image.open(img_path)

# convert image to numpy array
img = np.asarray(image)

# read deploy_cfg and model_cfg
deploy_cfg, model_cfg = load_config(deploy_cfg, model_cfg)

# build task and backend model
task_processor = build_task_processor(model_cfg, deploy_cfg, device)
model = task_processor.build_backend_model(backend_files)

# process input image
input_shape = get_input_shape(deploy_cfg)
model_inputs, _ = task_processor.create_input(img)

# do model inference
with torch.no_grad():
    result = model.test_step(model_inputs)

Environment

02/06 13:28:20 - mmengine - ←[4m←[97mINFO←[0m - **********Environmental information**********
02/06 13:28:26 - mmengine - INFO - sys.platform: win32
02/06 13:28:26 - mmengine - INFO - Python: 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:52:53) [MSC v.1927 64 bit (AMD64)]
02/06 13:28:26 - mmengine - INFO - CUDA available: False
02/06 13:28:26 - mmengine - INFO - numpy_random_seed: 2147483648
02/06 13:28:26 - mmengine - INFO - MSVC: Microsoft (R) C/C++ Optimizing Compiler Version 19.29.30139 for x64
02/06 13:28:26 - mmengine - INFO - GCC: n/a
02/06 13:28:26 - mmengine - INFO - PyTorch: 1.8.0+cpu
02/06 13:28:26 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 192829337
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
  - OpenMP 2019
  - CPU capability usage: AVX2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=C:/w/b/windows/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -DNDEBUG -DUSE_FBGEMM -DUSE_XNNPACK, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=0, USE_CUDNN=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON,

02/06 13:28:26 - mmengine - INFO - TorchVision: 0.9.0+cpu
02/06 13:28:26 - mmengine - INFO - OpenCV: 4.7.0
02/06 13:28:26 - mmengine - INFO - MMEngine: 0.5.0
02/06 13:28:26 - mmengine - INFO - MMCV: 2.0.0rc3
02/06 13:28:26 - mmengine - INFO - MMCV Compiler: MSVC 192829924
02/06 13:28:26 - mmengine - INFO - MMCV CUDA Compiler: not available
02/06 13:28:26 - mmengine - INFO - MMDeploy: 1.0.0rc1+cbddf5a
02/06 13:28:26 - mmengine - INFO -

02/06 13:28:26 - mmengine - INFO - **********Backend information**********
02/06 13:28:26 - mmengine - INFO - tensorrt:    None
02/06 13:28:26 - mmengine - INFO - ONNXRuntime: 1.8.1
02/06 13:28:26 - mmengine - INFO - ONNXRuntime-gpu:     None
02/06 13:28:26 - mmengine - INFO - ONNXRuntime custom ops:      Available
02/06 13:28:26 - mmengine - INFO - pplnn:       None
02/06 13:28:26 - mmengine - INFO - ncnn:        None
02/06 13:28:26 - mmengine - INFO - snpe:        None
02/06 13:28:26 - mmengine - INFO - openvino:    None
02/06 13:28:26 - mmengine - INFO - torchscript: 1.8.0+cpu
02/06 13:28:26 - mmengine - INFO - torchscript custom ops:      NotAvailable
02/06 13:28:26 - mmengine - INFO - rknn-toolkit:        None
02/06 13:28:26 - mmengine - INFO - rknn2-toolkit:       None
02/06 13:28:26 - mmengine - INFO - ascend:      None
02/06 13:28:26 - mmengine - INFO - coreml:      None
02/06 13:28:26 - mmengine - INFO - tvm: None
02/06 13:28:26 - mmengine - INFO -

02/06 13:28:26 - mmengine - INFO - **********Codebase information**********
02/06 13:28:26 - mmengine - INFO - mmdet:       3.0.0rc5
02/06 13:28:26 - mmengine - INFO - mmseg:       None
02/06 13:28:26 - mmengine - INFO - mmcls:       None
02/06 13:28:26 - mmengine - INFO - mmocr:       1.0.0rc5
02/06 13:28:26 - mmengine - INFO - mmedit:      None
02/06 13:28:26 - mmengine - INFO - mmdet3d:     None
02/06 13:28:26 - mmengine - INFO - mmpose:      None
02/06 13:28:26 - mmengine - INFO - mmrotate:    None
02/06 13:28:26 - mmengine - INFO - mmaction:    None

Error traceback

KeyError                                  Traceback (most recent call last)
Cell In[3], line 24
     21 model = task_processor.build_backend_model(backend_files)
     23 input_shape = get_input_shape(deploy_cfg)
---> 24 model_inputs, _ = task_processor.create_input(img)
     27 result = model.test_step(model_inputs)

File c:\Users\poopo\Desktop\Ikomia\envs\venvmmlab4\lib\site-packages\mmdeploy\codebase\mmocr\deploy\text_recognition.py:166, in TextRecognition.create_input(self, imgs, input_shape, data_preprocessor)
    164         data_ = dict(img_path=img, img_id=0)
    165     # build the data pipeline
--> 166     data_ = test_pipeline(data_)
    167     data.append(data_)
    169 data = pseudo_collate(data)

File c:\Users\poopo\Desktop\Ikomia\envs\venvmmlab4\lib\site-packages\mmcv\transforms\base.py:12, in BaseTransform.__call__(self, results)
      9 def __call__(self,
     10              results: Dict) -> Optional[Union[Dict, Tuple[List, List]]]:
---> 12     return self.transform(results)

File c:\Users\poopo\Desktop\Ikomia\envs\venvmmlab4\lib\site-packages\mmcv\transforms\wrappers.py:87, in Compose.transform(self, results)
     78 """Call function to apply transforms sequentially.
     79 
     80 Args:
   (...)
     84     dict or None: Transformed results.
     85 """
     86 for t in self.transforms:
---> 87     results = t(results)  # type: ignore
     88     if results is None:
     89         return None

File c:\Users\poopo\Desktop\Ikomia\envs\venvmmlab4\lib\site-packages\mmcv\transforms\base.py:12, in BaseTransform.__call__(self, results)
      9 def __call__(self,
     10              results: Dict) -> Optional[Union[Dict, Tuple[List, List]]]:
---> 12     return self.transform(results)

File c:\Users\poopo\Desktop\Ikomia\envs\venvmmlab4\lib\site-packages\mmocr\datasets\transforms\loading.py:81, in LoadImageFromFile.transform(self, results)
     67 """Functions to load image.
     68 
     69 Args:
     70     results (dict): Result dict from :obj:``mmcv.BaseDataset``.
     71 """
     72 """Functions to load image.
     73 
     74 Args:
   (...)
     78     dict: The dict contains loaded image and meta information.
     79 """
---> 81 filename = results['img_path']
     82 try:
     83     img_bytes = self.file_client.get(filename)

KeyError: 'img_path'
AllentDan commented 1 year ago

Thanks for reporting the bug. Will fix it ASAP.

AllentDan commented 1 year ago

Hi, the bug was fixed in pr #1721

allankouidri commented 1 year ago

Excellent! Thanks for the great work.

allankouidri commented 1 year ago

Hi @AllentDan, Thanks you for fixing the bug for text recognition.

There is the same error for the text detection models.

Let me know if I should open a new issue.

AllentDan commented 1 year ago

Hi @AllentDan, Thanks you for fixing the bug for text recognition.

There is the same error for the text detection models.

Let me know if I should open a new issue.

Hi, @allankouidri Will fix it soon. Thanks.