lxjhunan
commented
3 months ago 你好，遇到了一样的问题，不知道是否已经解决。
Open hymnli opened 4 months ago
lxjhunan
commented
3 months ago 你好，遇到了一样的问题，不知道是否已经解决。
hedgehoggy
commented
2 months ago Hello, I'm also experiencing this problem, has it been solved yet?
Checklist
Describe the bug
使用 deploy tools/torch2onnx.py 将grounding_dino 模型 groundingdino_swint_ogc_mmdet-822d7e9d.pth 转换为 onnx时报错
unexpected key in source state_dict: language_model.language_backbone.body.model.embeddings.position_ids
missing keys in source state_dict: dn_query_generator.label_embedding.weight
03/12 11:26:41 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 03/12 11:26:41 - mmengine - INFO - Export PyTorch model to ONNX: /home/srd-user/workspace/mmlab_prj/mmdetection/work_dirs/mmdeploy_models/mmdet/grounding_dino/end2end.onnx. /home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base_detr.py:85: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results). img_shape = [int(val) for val in img_shape] /home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base_detr.py:85: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! img_shape = [int(val) for val in img_shape] /home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/core/optimizers/function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! ys_shape = tuple(int(s) for s in ys.shape) /home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/layers/transformer/utils.py:167: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! output_h = math.ceil(input_h / stride_h) /home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/layers/transformer/utils.py:168: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! output_w = math.ceil(input_w / stride_w) /home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/layers/transformer/utils.py:169: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! pad_h = max((output_h - 1) stride_h + /home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/layers/transformer/utils.py:171: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! pad_w = max((output_w - 1) stride_w + /home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/layers/transformer/utils.py:177: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if pad_h > 0 or pad_w > 0: /home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/codebase/mmdet/models/backbones.py:189: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert L == H W, 'input feature has wrong size' /home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/backbones/swin.py:267: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! B = int(windows.shape[0] / (H W / window_size / window_size)) /home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/layers/transformer/utils.py:414: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert L == H W, 'input feature has wrong size' Traceback (most recent call last): File "tools/torch2onnx.py", line 85, in
main()
File "tools/torch2onnx.py", line 47, in main
torch2onnx(
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 356, in _wrap
return self.call_function(funcname, args, kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
return self.call_function_local(func_name, *args, *kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
return pipe_caller(args, kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in call
ret = func(*args, kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/pytorch2onnx.py", line 98, in torch2onnx
export(
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 356, in _wrap
return self.call_function(funcname, *args, *kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
return self.call_function_local(func_name, args, kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
return pipe_caller(*args, kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in call
ret = func(*args, *kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/onnx/export.py", line 138, in export
torch.onnx.export(
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/onnx/utils.py", line 516, in export
_export(
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/onnx/utils.py", line 1596, in _export
graph, params_dict, torch_out = _model_to_graph(
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/onnx/optimizer.py", line 27, in model_to_graph__custom_optimizer
graph, params_dict, torch_out = ctx.origin_func(args, kwargs)
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/onnx/utils.py", line 1135, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args)
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/onnx/utils.py", line 1011, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/onnx/utils.py", line 915, in _trace_and_get_graph_from_model
trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/jit/_trace.py", line 1285, in _get_trace_graph
outs = ONNXTracedModule(
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, kwargs)
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, kwargs)
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/jit/_trace.py", line 133, in forward
graph, out = torch._C._create_graph_by_tracing(
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/jit/_trace.py", line 124, in wrapper
outs.append(self.inner(trace_inputs))
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(args, kwargs)
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, kwargs)
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1508, in _slow_forward
result = self.forward(*input, kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/apis/onnx/export.py", line 123, in wrapper
return forward(*arg, *kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base_detr.py", line 89, in detection_transformerforward
return predict_impl(self, batch_inputs, data_samples, rescale)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/core/optimizers/function_marker.py", line 266, in g
rets = f(args, kwargs)
File "/home/srd-user/workspace/mmlab_prj/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base_detr.py", line 22, in __predict_impl
head_inputs_dict = self.forward_transformer(img_feats, data_samples)
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/detectors/grounding_dino.py", line 152, in forward_transformer
encoder_inputs_dict, decoder_inputs_dict = self.pre_transformer(
File "/home/srd-user/.conda/envs/mmlab_dep_cu118/lib/python3.8/site-packages/mmdet/models/detectors/deformable_detr.py", line 152, in pre_transformer
assert batch_data_samples is not None
AssertionError
(mmlab_dep_cu118) aaa-user@aaa-ubuntu-20:~/workspace/mmlab_prj/mmdeploy$
Reproduction
python tools/torch2onnx.py \ ${DEPLOY_CFG_PATH}/mmdet/detection/detection_onnxruntime_static.py \ ${MODEL_CFG_PATH}/grounding_dino/grounding_dino_swin-t_pretrain_obj365_goldg_cap4m.py \ ${MODEL_CHECKPOINT_PATH}/groundingdino_swint_ogc_mmdet-822d7e9d.pth \ ${INPUT_DIR}/images/firc_accident_5.jpg \ --work-dir ${WORK_DIR}/mmdeploy_models/mmdet/grounding_dino \ --device cpu \ --log-level INFO
Environment
Error traceback