search

IDEA-Research / DINO

[ICLR 2023] Official implementation of the paper "DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection"
Apache License 2.0
2.1k stars 230 forks source link

请问可否提供pth模型转onnx的脚本 #188

Open Zong-my opened 1 year ago

Zong-my commented 1 year ago

非常棒的工作，请问可否提供pth模型转onnx的脚本呢？

HaoZhang534 commented 1 year ago

你或许可以参考这个,DINO应该可以用相似的方法 https://github.com/IDEA-Research/detrex/issues/192

Sencc commented 6 months ago

请问你成功转换onnx了吗

Zong-my commented 6 months ago

请问你成功转换onnx了吗

未成功

Sencc commented 6 months ago

@Zong-my 我最近使用mmdeploy可以成功转换出onnx，但是它的检测速度比pytorch慢很多。

wuzujiong commented 6 months ago

@Zong-my 我最近使用mmdeploy可以成功转换出onnx，但是它的检测速度比pytorch慢很多。

我也是，速度差了10倍

Sencc commented 6 months ago

@wuzujiong 使用tensorrt加速后，速度正常

pof1423 commented 5 months ago

I succeeded in converting to onnx. I recommend converting it to torch.jit.trace first and then exporting it to onnx.

wuzujiong commented 4 months ago

@wuzujiong 使用tensorrt加速后，速度正常

可以介绍一下用的是什么环境导出来.engine 的吗？我在windows 上转engine 转不成功； PS D:\DL\mmdeploy-main\dino_crack_detections> trtexec --onnx=end2end.onnx --plugins=./mmdeploy_tensorrt_ops.dll --dumpProfile --separateProfileRun &&&& RUNNING TensorRT.trtexec [TensorRT v8400] # C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\bin\trtexec.exe --onnx=end2end.onnx --plugins=./mmdeploy_tensorrt_ops.dll --dumpProfile --separateProfileRun [02/25/2024-10:09:05] [I] === Model Options === [02/25/2024-10:09:05] [I] Format: ONNX [02/25/2024-10:09:05] [I] Model: end2end.onnx [02/25/2024-10:09:05] [I] Output: [02/25/2024-10:09:05] [I] === Build Options === [02/25/2024-10:09:05] [I] Max batch: explicit batch [02/25/2024-10:09:05] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default [02/25/2024-10:09:05] [I] minTiming: 1 [02/25/2024-10:09:05] [I] avgTiming: 8 [02/25/2024-10:09:05] [I] Precision: FP32 [02/25/2024-10:09:05] [I] LayerPrecisions: [02/25/2024-10:09:05] [I] Calibration: [02/25/2024-10:09:05] [I] Refit: Disabled [02/25/2024-10:09:05] [I] Sparsity: Disabled [02/25/2024-10:09:05] [I] Safe mode: Disabled [02/25/2024-10:09:05] [I] DirectIO mode: Disabled [02/25/2024-10:09:05] [I] Restricted mode: Disabled [02/25/2024-10:09:05] [I] Save engine: [02/25/2024-10:09:05] [I] Load engine: [02/25/2024-10:09:05] [I] Profiling verbosity: 0 [02/25/2024-10:09:05] [I] Tactic sources: Using default tactic sources [02/25/2024-10:09:05] [I] timingCacheMode: local [02/25/2024-10:09:05] [I] timingCacheFile: [02/25/2024-10:09:05] [I] Input(s)s format: fp32:CHW [02/25/2024-10:09:05] [I] Output(s)s format: fp32:CHW [02/25/2024-10:09:05] [I] Input build shapes: model [02/25/2024-10:09:05] [I] Input calibration shapes: model [02/25/2024-10:09:05] [I] === System Options === [02/25/2024-10:09:05] [I] Device: 0 [02/25/2024-10:09:05] [I] DLACore: [02/25/2024-10:09:05] [I] Plugins: ./mmdeploy_tensorrt_ops.dll [02/25/2024-10:09:05] [I] === Inference Options === [02/25/2024-10:09:05] [I] Batch: Explicit [02/25/2024-10:09:05] [I] Input inference shapes: model [02/25/2024-10:09:05] [I] Iterations: 10 [02/25/2024-10:09:05] [I] Duration: 3s (+ 200ms warm up) [02/25/2024-10:09:05] [I] Sleep time: 0ms [02/25/2024-10:09:05] [I] Idle time: 0ms [02/25/2024-10:09:05] [I] Streams: 1 [02/25/2024-10:09:05] [I] ExposeDMA: Disabled [02/25/2024-10:09:05] [I] Data transfers: Enabled [02/25/2024-10:09:05] [I] Spin-wait: Disabled [02/25/2024-10:09:05] [I] Multithreading: Disabled [02/25/2024-10:09:05] [I] CUDA Graph: Disabled [02/25/2024-10:09:05] [I] Separate profiling: Enabled [02/25/2024-10:09:05] [I] Time Deserialize: Disabled [02/25/2024-10:09:05] [I] Time Refit: Disabled [02/25/2024-10:09:05] [I] Skip inference: Disabled [02/25/2024-10:09:05] [I] Inputs: [02/25/2024-10:09:05] [I] === Reporting Options === [02/25/2024-10:09:05] [I] Verbose: Disabled [02/25/2024-10:09:05] [I] Averages: 10 inferences [02/25/2024-10:09:05] [I] Percentile: 99 [02/25/2024-10:09:05] [I] Dump refittable layers:Disabled [02/25/2024-10:09:05] [I] Dump output: Disabled [02/25/2024-10:09:05] [I] Profile: Enabled [02/25/2024-10:09:05] [I] Export timing to JSON file: [02/25/2024-10:09:05] [I] Export output to JSON file: [02/25/2024-10:09:05] [I] Export profile to JSON file: [02/25/2024-10:09:05] [I] [02/25/2024-10:09:05] [I] === Device Information === [02/25/2024-10:09:05] [I] Selected Device: NVIDIA GeForce RTX 3080 [02/25/2024-10:09:05] [I] Compute Capability: 8.6 [02/25/2024-10:09:05] [I] SMs: 68 [02/25/2024-10:09:05] [I] Compute Clock Rate: 1.785 GHz [02/25/2024-10:09:05] [I] Device Global Memory: 10239 MiB [02/25/2024-10:09:05] [I] Shared Memory per SM: 100 KiB [02/25/2024-10:09:05] [I] Memory Bus Width: 320 bits (ECC disabled) [02/25/2024-10:09:05] [I] Memory Clock Rate: 9.501 GHz [02/25/2024-10:09:05] [I] [02/25/2024-10:09:05] [I] TensorRT version: 8.4.0 [02/25/2024-10:09:05] [I] Loading supplied plugin library: ./mmdeploy_tensorrt_ops.dll [02/25/2024-10:09:06] [I] [TRT] [MemUsageChange] Init CUDA: CPU +519, GPU +0, now: CPU 12446, GPU 1346 (MiB) [02/25/2024-10:09:07] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 12651 MiB, GPU 1346 MiB [02/25/2024-10:09:08] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 13086 MiB, GPU 1468 MiB [02/25/2024-10:09:08] [I] Start parsing network model [02/25/2024-10:09:08] [I] [TRT] ---------------------------------------------------------------- [02/25/2024-10:09:08] [I] [TRT] Input filename: end2end.onnx [02/25/2024-10:09:08] [I] [TRT] ONNX IR version: 0.0.6 [02/25/2024-10:09:08] [I] [TRT] Opset version: 11 [02/25/2024-10:09:08] [I] [TRT] Producer name: pytorch [02/25/2024-10:09:08] [I] [TRT] Producer version: 1.11.0 [02/25/2024-10:09:08] [I] [TRT] Domain: [02/25/2024-10:09:08] [I] [TRT] Model version: 0 [02/25/2024-10:09:08] [I] [TRT] Doc string: [02/25/2024-10:09:08] [I] [TRT] ---------------------------------------------------------------- [02/25/2024-10:09:08] [W] [TRT] onnx2trt_utils.cpp:365: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. [02/25/2024-10:09:08] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin. [02/25/2024-10:09:08] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: [02/25/2024-10:09:08] [I] [TRT] Successfully created plugin: TRTInstanceNormalization [02/25/2024-10:09:08] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin. [02/25/2024-10:09:08] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: [02/25/2024-10:09:08] [I] [TRT] Successfully created plugin: TRTInstanceNormalization [02/25/2024-10:09:08] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin. [02/25/2024-10:09:08] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: [02/25/2024-10:09:08] [I] [TRT] Successfully created plugin: TRTInstanceNormalization [02/25/2024-10:09:08] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin. [02/25/2024-10:09:08] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace: [02/25/2024-10:09:08] [I] [TRT] Successfully created plugin: TRTInstanceNormalization [02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:748: While parsing node number 160 [Range -> "onnx::Reshape_864"]: [02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:749: --- Begin node --- [02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:750: input: "onnx::Range_8968" input: "onnx::Range_862" input: "onnx::Range_8968" output: "onnx::Reshape_864" name: "Range_160" op_type: "Range"

[02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:751: --- End node --- [02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:754: ERROR: ModelImporter.cpp:162 In function parseGraph: [6] Invalid Node - Range_160 All inputs to range should be initializers. [02/25/2024-10:09:08] [E] Failed to parse onnx file [02/25/2024-10:09:08] [I] Finish parsing network model [02/25/2024-10:09:08] [E] Parsing model failed [02/25/2024-10:09:08] [E] Failed to create engine from model. [02/25/2024-10:09:08] [E] Engine set up failed

shuchang0714 commented 4 months ago

@wuzujiong 使用tensorrt加速后，速度正常

请问下你dino转trt fp16的精度能对齐嘛

sherryhsy commented 4 months ago

@Zong-my 我最近使用mmdeploy可以成功转换出onnx，但是它的检测速度比pytorch慢很多。

请问是怎么转换的呢，我尝试maskdino转onnx，转换后输出结果是错误的

wulongjian commented 3 months ago

I succeeded in converting to onnx. I recommend converting it to torch.jit.trace first and then exporting it to onnx.

hello、Could ou share your conversion code and environment configuration? I didn't succeed just use torch.jit.trace(model, img)

pof1423 commented 2 months ago

I succeeded in converting to onnx. I recommend converting it to torch.jit.trace first and then exporting it to onnx.

hello、Could ou share your conversion code and environment configuration? I didn't succeed just use torch.jit.trace(model, img)

In ms_deform_attn.py change line 124 to 126 as output = ms_deform_attn_core_pytorch(value, input_spatial_shapes, sampling_locations, attention_weights)

and import ms_deform_attn_core_pytorch with adding from ..functions.ms_deform_attn_func import ms_deform_attn_core_pytorch

Ochre-amber commented 2 months ago

@wuzujiong @Sencc 你好，我尝试用mmdeploy导出onnx但是失败了

from mmdeploy.apis import torch2onnx
from mmdeploy.backend.sdk.export_info import export2SDK

img = 'demo/demo.jpg'
work_dir = 'mmdeploy_models/mmdet/onnx'
save_file = 'end2end.onnx'
deploy_cfg = '../mmdeploy/configs/mmdet/detection/detection_onnxruntime_dynamic.py'
model_cfg = 'configs/dino/dino-4scale_r50_8xb2-12e_coco.py'
model_checkpoint = ('dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth')
device = 'cuda'

# 1. convert model to onnx
torch2onnx(img, work_dir, save_file, deploy_cfg, model_cfg,
           model_checkpoint, device)

# 2. extract pipeline info for inference by MMDeploy SDK
export2SDK(deploy_cfg, model_cfg, work_dir, pth=model_checkpoint,
           device=device)

没有输出的模型

D:\Coding\anaconda\envs\tensoryolo\python.exe D:\Programs\last\project\mmdetection\to_onnx.py 
05/05 11:34:44 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
05/05 11:34:44 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth
05/05 11:34:46 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
05/05 11:34:46 - mmengine - INFO - Export PyTorch model to ONNX: mmdeploy_models/mmdet/onnx\end2end.onnx.
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:157: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  s[0] == input_img_h and s[1] == input_img_w for s in img_shape_list
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:156: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  same_shape_flag = all([
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\deformable_detr_layers.py:103: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, (H, W) in enumerate(spatial_shapes):
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\linspace.py:15: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if steps == 1:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\TensorShape.cpp:3527.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:335: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert (spatial_shapes[:, 0] * spatial_shapes[:, 1]).sum() == num_value
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:351: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 2:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:136: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  value_list = value.split([H_ * W_ for H_, W_ in value_spatial_shapes],
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:140: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for level, (H_, W_) in enumerate(value_spatial_shapes):
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:489: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, HW in enumerate(spatial_shapes):
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:490: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  H, W = HW
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\dino_layers.py:74: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 4:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\utils.py:80: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if coord_tensor.size(-1) == 2:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\utils.py:82: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif coord_tensor.size(-1) == 4:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:357: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif reference_points.shape[-1] == 4:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\dino_layers.py:101: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert reference_points.shape[-1] == 4
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:277: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  if len(query) == self.num_queries:
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:286: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  hidden_states=inter_states, references=list(references))
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:286: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  hidden_states=inter_states, references=list(references))
D:\Programs\last\project\mmdetection\mmdet\models\dense_heads\deformable_detr_head.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference.shape[-1] == 4:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\codebase\mmdet\models\dense_heads\detr_head.py:31: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  max_per_img = self.test_cfg.get('max_per_img', len(cls_scores[0]))
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)

Process finished with exit code -1073741819 (0xC0000005)

请问您是怎么导出的？

Sencc commented 2 months ago

@wuzujiong @Sencc 你好，我尝试用mmdeploy导出onnx但是失败了

from mmdeploy.apis import torch2onnx
from mmdeploy.backend.sdk.export_info import export2SDK

img = 'demo/demo.jpg'
work_dir = 'mmdeploy_models/mmdet/onnx'
save_file = 'end2end.onnx'
deploy_cfg = '../mmdeploy/configs/mmdet/detection/detection_onnxruntime_dynamic.py'
model_cfg = 'configs/dino/dino-4scale_r50_8xb2-12e_coco.py'
model_checkpoint = ('dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth')
device = 'cuda'

# 1. convert model to onnx
torch2onnx(img, work_dir, save_file, deploy_cfg, model_cfg,
           model_checkpoint, device)

# 2. extract pipeline info for inference by MMDeploy SDK
export2SDK(deploy_cfg, model_cfg, work_dir, pth=model_checkpoint,
           device=device)

没有输出的模型

D:\Coding\anaconda\envs\tensoryolo\python.exe D:\Programs\last\project\mmdetection\to_onnx.py 
05/05 11:34:44 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
05/05 11:34:44 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth
05/05 11:34:46 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
05/05 11:34:46 - mmengine - INFO - Export PyTorch model to ONNX: mmdeploy_models/mmdet/onnx\end2end.onnx.
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:157: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  s[0] == input_img_h and s[1] == input_img_w for s in img_shape_list
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:156: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  same_shape_flag = all([
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\deformable_detr_layers.py:103: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, (H, W) in enumerate(spatial_shapes):
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\linspace.py:15: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if steps == 1:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\TensorShape.cpp:3527.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:335: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert (spatial_shapes[:, 0] * spatial_shapes[:, 1]).sum() == num_value
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:351: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 2:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:136: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  value_list = value.split([H_ * W_ for H_, W_ in value_spatial_shapes],
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:140: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for level, (H_, W_) in enumerate(value_spatial_shapes):
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:489: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, HW in enumerate(spatial_shapes):
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:490: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  H, W = HW
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\dino_layers.py:74: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 4:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\utils.py:80: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if coord_tensor.size(-1) == 2:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\utils.py:82: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif coord_tensor.size(-1) == 4:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:357: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif reference_points.shape[-1] == 4:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\dino_layers.py:101: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert reference_points.shape[-1] == 4
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:277: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  if len(query) == self.num_queries:
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:286: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  hidden_states=inter_states, references=list(references))
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:286: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  hidden_states=inter_states, references=list(references))
D:\Programs\last\project\mmdetection\mmdet\models\dense_heads\deformable_detr_head.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference.shape[-1] == 4:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\codebase\mmdet\models\dense_heads\detr_head.py:31: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  max_per_img = self.test_cfg.get('max_per_img', len(cls_scores[0]))
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)

Process finished with exit code -1073741819 (0xC0000005)

请问您是怎么导出的？

要看一下onnx的版本，大于1.8才行

Sencc commented 2 months ago

@wuzujiong 使用tensorrt加速后，速度正常

请问下你dino转trt fp16的精度能对齐嘛

没有做具体测试，我感觉差不多

Ochre-amber commented 2 months ago

@Sencc onnx版本为1.16.0，似乎不是这个问题。

pip show onnx
Name: onnx
Version: 1.16.0
Summary: Open Neural Network Exchange
Home-page: 
Author: 
Author-email: ONNX Contributors <onnx-technical-discuss@lists.lfaidata.foundation>
License: Apache License v2.0
Location: d:\coding\anaconda\envs\tensoryolo\lib\site-packages
Requires: numpy, protobuf
Required-by: mmdeploy, onnx-graphsurgeon, onnx-simplifier

具体环境为：


05/05 09:53:24 - mmengine - INFO - **********Environmental information**********
05/05 09:53:28 - mmengine - INFO - sys.platform: win32
05/05 09:53:28 - mmengine - INFO - Python: 3.8.18 | packaged by conda-forge | (default, Dec 23 2023, 17:17:17) [MSC v.1929 64 bit (AMD64)]
05/05 09:53:28 - mmengine - INFO - CUDA available: True
05/05 09:53:28 - mmengine - INFO - MUSA available: False
05/05 09:53:28 - mmengine - INFO - numpy_random_seed: 2147483648
05/05 09:53:28 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 4060 Laptop GPU
05/05 09:53:28 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1
05/05 09:53:28 - mmengine - INFO - NVCC: Cuda compilation tools, release 12.1, V12.1.105
05/05 09:53:28 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.39.33523 版
05/05 09:53:28 - mmengine - INFO - GCC: n/a
05/05 09:53:28 - mmengine - INFO - PyTorch: 2.1.2
05/05 09:53:28 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 192930151
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX2
  - CUDA Runtime 12.1
  - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
  - CuDNN 8.8.1  (built against CUDA 12.0)
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.8.1, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.1.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

05/05 09:53:28 - mmengine - INFO - TorchVision: 0.16.2
05/05 09:53:28 - mmengine - INFO - OpenCV: 4.8.1
05/05 09:53:28 - mmengine - INFO - MMEngine: 0.10.4
05/05 09:53:28 - mmengine - INFO - MMCV: 2.1.0
05/05 09:53:28 - mmengine - INFO - MMCV Compiler: MSVC 192930148
05/05 09:53:28 - mmengine - INFO - MMCV CUDA Compiler: 12.1
05/05 09:53:28 - mmengine - INFO - MMDeploy: 1.0.0+bc75c9d
05/05 09:53:28 - mmengine - INFO -

05/05 09:53:28 - mmengine - INFO - **********Backend information**********
05/05 09:53:29 - mmengine - INFO - tensorrt:    8.6.1
05/05 09:53:29 - mmengine - INFO - tensorrt custom ops: Available
05/05 09:53:29 - mmengine - INFO - ONNXRuntime: 1.8.1
05/05 09:53:29 - mmengine - INFO - ONNXRuntime-gpu:     None
05/05 09:53:29 - mmengine - INFO - ONNXRuntime custom ops:      Available
05/05 09:53:29 - mmengine - INFO - pplnn:       None
05/05 09:53:29 - mmengine - INFO - ncnn:        None
05/05 09:53:29 - mmengine - INFO - snpe:        None
05/05 09:53:29 - mmengine - INFO - openvino:    None
05/05 09:53:29 - mmengine - INFO - torchscript: 2.1.2+cu121
05/05 09:53:29 - mmengine - INFO - torchscript custom ops:      NotAvailable
05/05 09:53:29 - mmengine - INFO - rknn-toolkit:        None
05/05 09:53:29 - mmengine - INFO - rknn-toolkit2:       None
05/05 09:53:29 - mmengine - INFO - ascend:      None
05/05 09:53:29 - mmengine - INFO - coreml:      None
05/05 09:53:29 - mmengine - INFO - tvm: None
05/05 09:53:29 - mmengine - INFO - vacc:        None
05/05 09:53:29 - mmengine - INFO -

05/05 09:53:29 - mmengine - INFO - **********Codebase information**********
05/05 09:53:29 - mmengine - INFO - mmdet:       3.3.0
05/05 09:53:29 - mmengine - INFO - mmseg:       None
05/05 09:53:29 - mmengine - INFO - mmcls:       None
05/05 09:53:29 - mmengine - INFO - mmocr:       None
05/05 09:53:29 - mmengine - INFO - mmedit:      None
05/05 09:53:29 - mmengine - INFO - mmdet3d:     None
05/05 09:53:29 - mmengine - INFO - mmpose:      None
05/05 09:53:29 - mmengine - INFO - mmrotate:    None
05/05 09:53:29 - mmengine - INFO - mmaction:    None
05/05 09:53:29 - mmengine - INFO - mmrazor:     None

请问可能是环境配置问题嘛？您的环境是什么样的？

Sencc commented 2 months ago

@Ochre-amber 这是我的环境

`05/07 11:06:53 - mmengine - INFO - TorchVision: 0.14.1+cu117 05/07 11:06:53 - mmengine - INFO - OpenCV: 4.9.0 05/07 11:06:53 - mmengine - INFO - MMEngine: 0.10.2 05/07 11:06:53 - mmengine - INFO - MMCV: 2.1.0 05/07 11:06:53 - mmengine - INFO - MMCV Compiler: GCC 9.3 05/07 11:06:53 - mmengine - INFO - MMCV CUDA Compiler: 11.7 05/07 11:06:53 - mmengine - INFO - MMDeploy: 1.3.1+44ebd17 05/07 11:06:53 - mmengine - INFO -

05/07 11:06:53 - mmengine - INFO - **Backend information** 05/07 11:06:54 - mmengine - INFO - tensorrt: 8.6.1 05/07 11:06:54 - mmengine - INFO - tensorrt custom ops: Available 05/07 11:06:54 - mmengine - INFO - ONNXRuntime: None 05/07 11:06:54 - mmengine - INFO - ONNXRuntime-gpu: 1.16.3 05/07 11:06:54 - mmengine - INFO - ONNXRuntime custom ops: Available 05/07 11:06:54 - mmengine - INFO - pplnn: None 05/07 11:06:54 - mmengine - INFO - ncnn: None 05/07 11:06:54 - mmengine - INFO - snpe: None 05/07 11:06:54 - mmengine - INFO - openvino: None 05/07 11:06:54 - mmengine - INFO - torchscript: 1.13.1+cu117 05/07 11:06:54 - mmengine - INFO - torchscript custom ops: NotAvailable 05/07 11:06:54 - mmengine - INFO - rknn-toolkit: None 05/07 11:06:54 - mmengine - INFO - rknn-toolkit2: None 05/07 11:06:54 - mmengine - INFO - ascend: None 05/07 11:06:54 - mmengine - INFO - coreml: None 05/07 11:06:54 - mmengine - INFO - tvm: None 05/07 11:06:54 - mmengine - INFO - vacc: None 05/07 11:06:54 - mmengine - INFO - `