[Bug] can't to convert YOLOX tiny model to onnx

xddun commented 1 year ago

Checklist

[X] I have searched related issues but cannot get the expected help.
[X] 2. I have read the FAQ documentation but cannot get the expected help.
[X] 3. The bug has not been fixed in the latest version.

Describe the bug

i want to convert YOLOX tiny model to onnx, there are no changes to the program, BUT the transformation of the model cannot be completed.

looking forward to your reply.

Reproduction

run:

python ./tools/deploy.py configs/mmdet/detection/detection_onnxruntime_static.py /ssd/xiedong/workplace/mmdetection/configs/yolox/yolox_tiny_8x8_300e_coco.py /ssd/xiedong/workplace/mmdetection/weights/yolox_tiny_8x8_300e_coco_20211124_171234-b4047906.pth /ssd/xiedong/workplace/mmdetection/demo/demo.jpg  --work-dir work_dir1 --device cpu

Environment

software version:

(py37c) xiedong@gpu16:/ssd/xiedong/workplace/mmdeploy$ pip list
Package                      Version              Editable project location
---------------------------- -------------------- ---------------------------------------
absl-py                      1.2.0
addict                       2.4.0
albumentations               1.0.1
aliyun-log-python-sdk        0.7.13
anyio                        3.6.1
astor                        0.8.1
astunparse                   1.6.3
attrs                        22.1.0
backcall                     0.2.0
backports.zoneinfo           0.2.1
baidu-aip                    4.16.7
beautifulsoup4               4.11.1
bottle                       0.12.23
bypy                         1.8
cachetools                   5.2.0
certifi                      2022.6.15
chardet                      5.0.0
charset-normalizer           2.1.1
click                        8.1.3
codecov                      2.1.12
colorama                     0.4.5
coloredlogs                  15.0.1
commonmark                   0.9.1
conan                        1.52.0
coverage                     6.4.4
cssselect2                   0.7.0
cycler                       0.11.0
Cython                       0.29.32
dateparser                   1.1.1
decorator                    5.1.1
deepface                     0.0.75
dill                         0.3.5.1
distro                       1.6.0
easydict                     1.10
elastic-transport            8.4.0
elasticsearch                8.4.0
faiss-cpu                    1.7.2
fastapi                      0.80.0
fasteners                    0.18
filelock                     3.8.0
fire                         0.4.0
flake8                       5.0.4
Flask                        2.2.2
flatbuffers                  1.12
fonttools                    4.36.0
gast                         0.4.0
gdown                        4.5.1
google-auth                  2.11.0
google-auth-oauthlib         0.4.6
google-pasta                 0.2.0
graphviz                     0.8.4
grpcio                       1.48.0
h11                          0.13.0
h5py                         3.7.0
humanfriendly                10.0
idna                         3.3
imageio                      2.21.1
importlib-metadata           4.12.0
infomap                      2.6.0
iniconfig                    1.1.1
interrogate                  1.5.0
ipython                      7.34.0
isort                        4.3.21
itsdangerous                 2.1.2
jedi                         0.18.1
Jinja2                       3.1.2
jmespath                     1.0.1
joblib                       1.1.0
keras                        2.9.0
Keras-Preprocessing          1.1.2
kiwisolver                   1.4.4
libclang                     14.0.6
lxml                         4.9.1
Markdown                     3.4.1
MarkupSafe                   2.1.1
matplotlib                   3.5.3
matplotlib-inline            0.1.6
mccabe                       0.7.0
mmcls                        0.23.2               /ssd/xiedong/workplace/mmclassification
mmcv-full                    1.6.1
mmdeploy                     0.7.0                /ssd/xiedong/workplace/mmdeploy
mmdet                        2.25.3               /ssd/xiedong/workplace/mmdetection
mmengine                     0.1.0
mmrazor                      0.3.1                /ssd/xiedong/workplace/mmrazor
mmsegmentation               0.29.0               /ssd/xiedong/workplace/mmsegmentation
mmyolo                       0.1.1                /ssd/xiedong/workplace/mmyolo
MNN                          2.0.0
model-index                  0.1.11
mpmath                       1.2.1
mtcnn                        0.1.1
multiprocess                 0.70.13
mxnet-native                 1.8.0.post0
networkx                     2.6.3
nmslib                       2.1.1
node-semver                  0.6.1
numpy                        1.21.6
nvidia-pyindex               1.0.9
oauthlib                     3.2.0
onnx                         1.12.0
onnx-simplifier              0.4.7
onnx-tool                    0.2.14
onnxruntime                  1.8.1
opencv-python                4.6.0.66
openmim                      0.2.1
opt-einsum                   3.3.0
ordered-set                  4.1.0
packaging                    21.3
pandas                       1.3.5
parso                        0.8.3
patch-ng                     1.17.4
pexpect                      4.8.0
pickleshare                  0.7.5
Pillow                       9.2.0
pip                          22.1.2
pluggy                       1.0.0
pluginbase                   1.0.1
prettytable                  3.5.0
prompt-toolkit               3.0.30
protobuf                     3.19.4
psutil                       5.9.1
ptyprocess                   0.7.0
py                           1.11.0
pyasn1                       0.4.8
pyasn1-modules               0.2.8
pybind11                     2.6.1
pycocotools                  2.0.4
pycodestyle                  2.9.1
pydantic                     1.9.2
pyflakes                     2.5.0
pyflann                      1.6.14
Pygments                     2.13.0
PyJWT                        2.5.0
pyparsing                    3.0.9
PySocks                      1.7.1
pytest                       7.1.2
python-dateutil              2.8.2
python-multipart             0.0.5
pytz                         2022.2.1
pytz-deprecation-shim        0.1.0.post0
PyWavelets                   1.3.0
PyYAML                       6.0
qudida                       0.0.4
regex                        2022.3.2
reportlab                    3.6.11
requests                     2.28.1
requests-oauthlib            1.3.1
requests-toolbelt            0.9.1
retina-face                  0.0.12
rich                         12.5.1
rsa                          4.9
scikit-image                 0.19.3
scikit-learn                 1.0.2
scipy                        1.7.3
seaborn                      0.11.2
setuptools                   61.2.0
six                          1.16.0
sniffio                      1.2.0
soupsieve                    2.3.2.post1
starlette                    0.19.1
svglib                       1.4.1
sympy                        1.10.1
tabulate                     0.8.10
tensorboard                  2.9.1
tensorboard-data-server      0.6.1
tensorboard-plugin-wit       1.8.1
tensorflow                   2.9.1
tensorflow-estimator         2.9.0
tensorflow-io-gcs-filesystem 0.26.0
termcolor                    1.1.0
terminaltables               3.1.10
thop                         0.1.1.post2207130030
threadpoolctl                3.1.0
tifffile                     2021.11.2
tinycss2                     1.1.1
toml                         0.10.2
tomli                        2.0.1
torch                        1.12.1+cu116
torchaudio                   0.12.1+cu116
torchvision                  0.13.1+cu116
tqdm                         4.64.0
traitlets                    5.3.0
typing_extensions            4.3.0
tzdata                       2022.2
tzlocal                      4.2
urllib3                      1.26.12
uvicorn                      0.18.3
wcwidth                      0.2.5
webencodings                 0.5.1
Werkzeug                     2.2.2
wheel                        0.37.1
wrapt                        1.14.1
xdoctest                     1.0.2
xpinyin                      0.7.6
yapf                         0.32.0
zipp                         3.8.1


### Error traceback

```Shell
output:

(py37c) xiedong@gpu16:/ssd/xiedong/workplace/mmdeploy$ python ./tools/deploy.py configs/mmdet/detection/detection_onnxruntime_static.py /ssd/xiedong/workplace/mmdetection/configs/yolox/yolox_tiny_8x8_300e_coco.py /ssd/xiedong/workplace/mmdetection/weights/yolox_tiny_8x8_300e_coco_20211124_171234-b4047906.pth /ssd/xiedong/workplace/mmdetection/demo/demo.jpg  --work-dir work_dir1 --device cpu
2022-11-02 16:18:26,764 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
load checkpoint from local path: /ssd/xiedong/workplace/mmdetection/weights/yolox_tiny_8x8_300e_coco_20211124_171234-b4047906.pth
2022-11-02 16:18:28,877 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
2022-11-02 16:18:28,877 - mmdeploy - INFO - Export PyTorch model to ONNX: work_dir1/end2end.onnx.
/ssd/xiedong/workplace/mmdeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/ssd/xiedong/workplace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py:24: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  img_shape = [int(val) for val in img_shape]
/ssd/xiedong/workplace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py:24: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  img_shape = [int(val) for val in img_shape]
Process Process-2:
Traceback (most recent call last):
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/apis/pytorch2onnx.py", line 105, in torch2onnx
    optimize=optimize)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 356, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
    return self.call_function_local(func_name, *args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
    return pipe_caller(*args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/apis/onnx/export.py", line 132, in export
    verbose=verbose)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/onnx/__init__.py", line 365, in export
    export_modules_as_functions,
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/onnx/utils.py", line 178, in export
    export_modules_as_functions=export_modules_as_functions,
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/onnx/utils.py", line 1084, in _export
    dynamic_axes=dynamic_axes,
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 379, in wrapper
    return self.func(self, *args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/apis/onnx/optimizer.py", line 10, in model_to_graph__custom_optimizer
    graph, params_dict, torch_out = ctx.origin_func(*args, **kwargs)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/onnx/utils.py", line 727, in _model_to_graph
    graph, params, torch_out, module = _create_jit_graph(model, args)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/onnx/utils.py", line 602, in _create_jit_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/onnx/utils.py", line 518, in _trace_and_get_graph_from_model
    model, args, strict=False, _force_outplace=False, _return_inputs_states=True
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/jit/_trace.py", line 1175, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/jit/_trace.py", line 132, in forward
    self._force_outplace,
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/jit/_trace.py", line 118, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 379, in wrapper
    return self.func(self, *args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py", line 70, in base_detector__forward
    return __forward_impl(ctx, self, img, img_metas=img_metas, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/core/optimizers/function_marker.py", line 261, in g
    rets = f(*args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py", line 26, in __forward_impl
    return self.simple_test(img, img_metas, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 379, in wrapper
    return self.func(self, *args, **kwargs)
  File "/ssd/xiedong/workplace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py", line 27, in single_stage_detector__simple_test
    feat = self.extract_feat(img)
  File "/ssd/xiedong/workplace/mmdetection/mmdet/models/detectors/single_stage.py", line 45, in extract_feat
    x = self.neck(x)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/ssd/xiedong/miniconda3/envs/py37c/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1118, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/ssd/xiedong/workplace/mmdetection/mmdet/models/necks/yolox_pafpn.py", line 139, in forward
    torch.cat([upsample_feat, feat_low], 1))
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 36 but got size 35 for tensor number 1 in the list.
2022-11-02 16:18:34,918 - mmdeploy - ERROR - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.

xddun commented 1 year ago

mmcls model to onnx model is available to me

grimoire commented 1 year ago

Try detection_onnxruntime_static.py

grimoire commented 1 year ago

@hanrui1sensetime static shape yolox failed.

xddun commented 1 year ago

thank you for your attention, i attempt to test the model in the mobile terminal， I want also to use the RTMDet (your mmyolo project launched) model , but it is failed too, look forward to see the tutorial update.

sebos123 commented 1 year ago

Hi, any update on this? I also get an error with dimension mismatch when exporting yolox.

grimoire commented 1 year ago

@xddun @sebos123 Try dynamic config. We are working on fixing the static one.

hanrui1sensetime commented 1 year ago

Hi, all: You should ensure your image has a shape in multiples of 32 if you deploy YOLOX and RTMDet model with static deploy config. YOLOX and RTMDet requires the image shape in multiples of 32. Other model requires the shape in multiples of a constant will face the same problems. So we recommend you use dynamic config.

grimoire commented 1 year ago

@hanrui1sensetime Why should we limit the image shape when using static shape? Preprocess should do the resize for us.

hanrui1sensetime commented 1 year ago

It is solved after #1436 and #1439. Please update mmdeploy and have a try.

open-mmlab / mmdeploy