[X] I have searched related issues but cannot get the expected help.
[X] 2. I have read the FAQ documentation but cannot get the expected help.
[x] 3. The bug has not been fixed in the latest version.
Describe the bug
I tried to convert RTMDet into ONNX version. But I keep receive this warning
> 03/07 03:07:27 - mmengine - INFO - Execute onnx optimize passes.
> 03/07 03:07:27 - mmengine - WARNING - Can not optimize model, please build torchscipt extension.
> More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md
At beginning, I ignored warning and served converted onnx model on CPU. I encounter high CPU usage. Whole throughput is quite low compared with Yolov5-m.
May I know how could optimise model to further improve the throughput?
03/07 03:07:11 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/07 03:07:11 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/07 03:07:18 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
03/07 03:07:23 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/07 03:07:23 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: /workspace/jupyter_notebooks/chimera-mlpipeline-tis-safety-ds/pipelines/food-watermark-detector-od/sandboxjob/rtmdet-train-datasetv2/mmdetection/work_dirs/rtmdet_tiny_8xb32-300e_coco_competitor_refine/epoch_30.pth
03/07 03:07:23 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
03/07 03:07:23 - mmengine - INFO - Export PyTorch model to ONNX: mmdeploy_models/mmdet/onnx/end2end.onnx.
/workspace/jupyter_notebooks/chimera-mlpipeline-tis-safety-ds/pipelines/food-watermark-detector-od/sandboxjob/rtmdet-train-datasetv2/mmdeploy/mmdeploy/core/optimizers/function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ys_shape = tuple(int(s) for s in ys.shape)
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3526.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/workspace/jupyter_notebooks/chimera-mlpipeline-tis-safety-ds/pipelines/food-watermark-detector-od/sandboxjob/rtmdet-train-datasetv2/mmdeploy/mmdeploy/mmcv/ops/nms.py:285: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
/workspace/jupyter_notebooks/chimera-mlpipeline-tis-safety-ds/pipelines/food-watermark-detector-od/sandboxjob/rtmdet-train-datasetv2/mmdeploy/mmdeploy/mmcv/ops/nms.py:286: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
/workspace/jupyter_notebooks/chimera-mlpipeline-tis-safety-ds/pipelines/food-watermark-detector-od/sandboxjob/rtmdet-train-datasetv2/mmdeploy/mmdeploy/pytorch/functions/topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
k = torch.tensor(k, device=input.device, dtype=torch.long)
/workspace/jupyter_notebooks/chimera-mlpipeline-tis-safety-ds/pipelines/food-watermark-detector-od/sandboxjob/rtmdet-train-datasetv2/mmdeploy/mmdeploy/mmcv/ops/nms.py:45: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
score_threshold = float(score_threshold)
/workspace/jupyter_notebooks/chimera-mlpipeline-tis-safety-ds/pipelines/food-watermark-detector-od/sandboxjob/rtmdet-train-datasetv2/mmdeploy/mmdeploy/mmcv/ops/nms.py:46: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
iou_threshold = float(iou_threshold)
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/mmcv/ops/nms.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert boxes.size(1) == 4
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/mmcv/ops/nms.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert boxes.size(0) == scores.size(0)
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:5857: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
warnings.warn(
03/07 03:07:27 - mmengine - INFO - Execute onnx optimize passes.
03/07 03:07:27 - mmengine - WARNING - Can not optimize model, please build torchscipt extension.
More details: https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/experimental/onnx_optimizer.md
03/07 03:07:28 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
03/07 03:07:29 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 1.061442947425384e-11 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -1.0665759725067558e-12 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 4.991658670606969e-14 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -4.04090877512836e-13 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 4.585053048344889e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 8.552871122446959e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 9.144637047597826e-09 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 2.4172816637246797e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 2.6420107768210244e-10 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -1.3031441203903427e-10 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -1.932203730348192e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 4.090862404382278e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -5.7430895594734466e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 1.3804506870940259e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -3.773499557269133e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 3.9884739067019837e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -4.245174523020978e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -1.5679363585263673e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 1.227207846454803e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -7.18615495998165e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 2.3395423376371127e-09 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -6.355934889512582e-09 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 7.902974630269455e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 4.10794740446363e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -2.044063229789117e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -9.02813965808491e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 2.7914245137594662e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -2.3122439074541035e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 2.5235621592401003e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -2.286071243418064e-08 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 2.6505876604687728e-08 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:43: UserWarning: the float32 number 1.5504121320120134e-09 will be truncated to 1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(pos_min, min_positive_val))
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/onnxconverter_common/float16.py:53: UserWarning: the float32 number -6.571862165571929e-09 will be truncated to -1e-07
warnings.warn("the float32 number {} will be truncated to {}".format(neg_max, -min_positive_val))
03/07 03:07:29 - mmengine - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend
03/07 03:07:29 - mmengine - INFO - visualize onnxruntime model start.
03/07 03:07:41 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/07 03:07:41 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/07 03:07:41 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/07 03:07:41 - mmengine - WARNING - The library of onnxruntime custom ops doesnot exist:
03/07 03:07:42 - mmengine - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
03/07 03:07:43 - mmengine - INFO - visualize onnxruntime model success.
03/07 03:07:43 - mmengine - INFO - visualize pytorch model start.
03/07 03:07:54 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
03/07 03:07:54 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: /workspace/jupyter_notebooks/chimera-mlpipeline-tis-safety-ds/pipelines/food-watermark-detector-od/sandboxjob/rtmdet-train-datasetv2/mmdetection/work_dirs/rtmdet_tiny_8xb32-300e_coco_competitor_refine/epoch_30.pth
/workspace/.pyenv/versions/mmdet_deploy/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3526.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
03/07 03:07:55 - mmengine - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
03/07 03:07:56 - mmengine - INFO - visualize pytorch model success.
03/07 03:07:56 - mmengine - INFO - All process success.
xanderread
commented
6 months ago
Checklist
Describe the bug
I tried to convert RTMDet into ONNX version. But I keep receive this warning
At beginning, I ignored warning and served converted onnx model on CPU. I encounter high CPU usage. Whole throughput is quite low compared with Yolov5-m.
May I know how could optimise model to further improve the throughput?
Reproduction
python tools/deploy.py configs/mmdet/detection/detection_onnxruntime-fp16_dynamic.py rtmdet_tiny_8xb32-300e_coco_competitor_refine.py rtmdet_tiny_8xb32-300e_coco_competitor_refine/epoch_30.pth image.jpg --work-dir mmdeploy_models/mmdet/onnx --device cpu --show --dump-infoEnvironment
Error traceback