Search before asking

[X] I have searched the DAMO-YOLO issues and found no similar feature requests.

Description

I tried to export the S model to Torch script by adding the following code to the function main() in converter.py

creating a CPU tensor

cpu_tensor = torch.rand(1, 3, 640, 640)

# x = torch.rand(1, 3, args.img_size, args.img_size)
x = cpu_tensor.to(device)

net = model.eval()

# mod = torch.jit.trace(net, x)
mod = torch.jit.trace(net, x, check_trace=False)

mod.save(torchscript_name)

When I run it "python tools/converter.py -f configs/damoyolo_tinynasL25_S.py -c ./damoyolo_tinynasL25_S_460.pth --batch_size 1 --img_size 640 "

I got the following error:

python tools/converter.py -f configs/damoyolo_tinynasL25_S.py -c ./damoyolo_tinynasL25_S_460.pth --batch_size 1 --img_size 640 --device cpu

2023-05-04 16:51:14.895 | INFO | main:main:180 - args value: Namespace(batch_size=1, benchmark=False, ckpt='./damoyolo_tinynasL25_S_460.pth', conf_thres=0.05, config_file='configs/damoyolo_tinynasL25_S.py', device='cpu', end2end=False, img_size=640, input='images', iou_thres=0.65, mode='onnx', opset=11, opts=[], ort=False, output='output', topk_all=100, trt=False, trt_eval=False, trt_type='fp32', with_preprocess=False) /home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/tensor.py:593: RuntimeWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results). 'incorrect results).', category=RuntimeWarning) 2023-05-04 16:51:16.535 | ERROR | main::294 - An error has been caught in function '', process 'MainProcess' (109183), thread 'MainThread' (139940674265728): Traceback (most recent call last):

File "tools/converter.py", line 294, in main() └ <function main at 0x7f46549107a0>

File "tools/converter.py", line 219, in main mod = torch.jit.trace(net, x, check_trace=False) │ │ │ │ └ tensor([[[[0.7899, 0.3109, 0.0514, ..., 0.7934, 0.9191, 0.2744], │ │ │ │ [0.7974, 0.1521, 0.0419, ..., 0.9119, 0.9891, 0.... │ │ │ └ Detector( │ │ │ (backbone): TinyNAS( │ │ │ (block_list): ModuleList( │ │ │ (0): Focus( │ │ │ (conv): ConvBNAct( │ │ │ (conv):... │ │ └ <function trace at 0x7f4668565050> │ └ <module 'torch.jit' from '/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/jit/init.py'> └ <module 'torch' from '/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/init.py'>

File "/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/jit/_trace.py", line 742, in trace _module_class, └ None File "/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/jit/_trace.py", line 940, in trace_module _force_outplace, └ False File "/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/nn/modules/module.py", line 725, in _call_impl result = self._slow_forward(*input, *kwargs) │ │ │ └ {} │ │ └ (tensor([[[[0.7899, 0.3109, 0.0514, ..., 0.7934, 0.9191, 0.2744], │ │ [0.7974, 0.1521, 0.0419, ..., 0.9119, 0.9891, 0... │ └ <function Module._slow_forward at 0x7f466ad6b830> └ Detector( (backbone): TinyNAS( (block_list): ModuleList( (0): Focus( (conv): ConvBNAct( (conv):... File "/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/nn/modules/module.py", line 709, in _slow_forward result = self.forward(input, **kwargs) │ │ │ └ {} │ │ └ (tensor([[[[0.7899, 0.3109, 0.0514, ..., 0.7934, 0.9191, 0.2744], │ │ [0.7974, 0.1521, 0.0419, ..., 0.9119, 0.9891, 0... │ └ <function Detector.forward at 0x7f4654941710> └ Detector( (backbone): TinyNAS( (block_list): ModuleList( (0): Focus( (conv): ConvBNAct( (conv):...

File "/home/bright/development/github/DAMO-YOLO/damo/detectors/detector.py", line 55, in forward feature_outs = self.backbone(images.tensors) # list of tensor │ │ └ tensor([[[[0.7899, 0.3109, 0.0514, ..., 0.7934, 0.9191, 0.2744], │ │ [0.7974, 0.1521, 0.0419, ..., 0.9119, 0.9891, 0.... │ └ <damo.structures.image_list.ImageList object at 0x7f465246d590> └ Detector( (backbone): TinyNAS( (block_list): ModuleList( (0): Focus( (conv): ConvBNAct( (conv):...

File "/home/bright/development/github/DAMO-YOLO/damo/base_models/backbones/tinynas_res.py", line 203, in forward output = block(output) │ └ tensor([[[[0.7899, 0.3109, 0.0514, ..., 0.7934, 0.9191, 0.2744], │ [0.7974, 0.1521, 0.0419, ..., 0.9119, 0.9891, 0.... └ Focus( (conv): ConvBNAct( (conv): Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn...

File "/home/bright/development/github/DAMO-YOLO/damo/base_models/core/ops.py", line 182, in forward return self.conv(x) │ └ tensor([[[[0.7899, 0.0514, 0.6211, ..., 0.8501, 0.8074, 0.9191], │ [0.7300, 0.2605, 0.5259, ..., 0.7807, 0.3813, 0.... └ Focus( (conv): ConvBNAct( (conv): Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn...

File "/home/bright/development/github/DAMO-YOLO/damo/base_models/core/ops.py", line 105, in forward x = self.conv(x) │ └ tensor([[[[0.7899, 0.0514, 0.6211, ..., 0.8501, 0.8074, 0.9191], │ [0.7300, 0.2605, 0.5259, ..., 0.7807, 0.3813, 0.... └ ConvBNAct( (conv): Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(32, ep...

File "/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/nn/modules/module.py", line 725, in _call_impl result = self._slow_forward(*input, *kwargs) │ │ │ └ {} │ │ └ (tensor([[[[0.7899, 0.0514, 0.6211, ..., 0.8501, 0.8074, 0.9191], │ │ [0.7300, 0.2605, 0.5259, ..., 0.7807, 0.3813, 0... │ └ <function Module._slow_forward at 0x7f466ad6b830> └ Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) File "/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/nn/modules/module.py", line 709, in _slow_forward result = self.forward(input, **kwargs) │ │ │ └ {} │ │ └ (tensor([[[[0.7899, 0.0514, 0.6211, ..., 0.8501, 0.8074, 0.9191], │ │ [0.7300, 0.2605, 0.5259, ..., 0.7807, 0.3813, 0... │ └ <function Conv2d.forward at 0x7f46686c3b00> └ Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) File "/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 423, in forward return self._conv_forward(input, self.weight) │ │ │ └ Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) │ │ └ tensor([[[[0.7899, 0.0514, 0.6211, ..., 0.8501, 0.8074, 0.9191], │ │ [0.7300, 0.2605, 0.5259, ..., 0.7807, 0.3813, 0.... │ └ <function Conv2d._conv_forward at 0x7f46686c3a70> └ Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) File "/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 420, in _conv_forward self.padding, self.dilation, self.groups) │ │ │ │ │ └ 1 │ │ │ │ └ Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) │ │ │ └ (1, 1) │ │ └ Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) │ └ (1, 1) └ Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)

RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

Use case

I would like to use the DAMO_YOLO model with ncnn framework. I have got some unsupported ops error when I convert the onnx model to ncnn model. Another better way is to use PNNX to convert the pytorch script model to NCNN model and this is why I think this feature is required.

Additional

No response

Are you willing to submit a PR?

[ ] Yes I'd like to help by submitting a PR!

I got the following error if there is no "--device cpu"

python tools/converter.py -f configs/damoyolo_tinynasL25_S.py -c ./damoyolo_tinynasL25_S_460.pth --batch_size 1 --img_size 640 2023-05-04 16:49:49.599 | INFO | main:main:180 - args value: Namespace(batch_size=1, benchmark=False, ckpt='./damoyolo_tinynasL25_S_460.pth', conf_thres=0.05, config_file='configs/damoyolo_tinynasL25_S.py', device='0', end2end=False, img_size=640, input='images', iou_thres=0.65, mode='onnx', opset=11, opts=[], ort=False, output='output', topk_all=100, trt=False, trt_eval=False, trt_type='fp32', with_preprocess=False) /home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/tensor.py:593: RuntimeWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results). 'incorrect results).', category=RuntimeWarning) /home/bright/development/github/DAMO-YOLO/damo/base_models/heads/zero_head.py:338: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! x_range = (torch.arange(0, int(w), dtype=dtype, /home/bright/development/github/DAMO-YOLO/damo/base_models/heads/zero_head.py:340: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! yrange = (torch.arange(0, int(h), dtype=dtype, /home/bright/development/github/DAMO-YOLO/damo/utils/boxes.py:120: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! output = [None for in range(batch_size)] /home/bright/development/github/DAMO-YOLO/damo/utils/boxes.py:121: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! for i in range(batch_size): /home/bright/development/github/DAMO-YOLO/damo/utils/boxes.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if not bbox_preds[i].size(0): /home/bright/development/github/DAMO-YOLO/damo/utils/boxes.py:49: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if multi_bboxes.shape[1] > 4: /home/bright/development/github/DAMO-YOLO/damo/structures/bounding_box.py:21: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. bbox = torch.as_tensor(bbox, dtype=torch.float32, device=device) /home/bright/development/github/DAMO-YOLO/damo/structures/bounding_box.py:25: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if bbox.size(-1) != 4: /home/bright/development/github/DAMO-YOLO/damo/structures/bounding_box.py:238: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! s += 'num_boxes={}, '.format(len(self)) 2023-05-04 16:49:51.781 | ERROR | main::294 - An error has been caught in function '', process 'MainProcess' (108860), thread 'MainThread' (139827662176896): Traceback (most recent call last):

File "tools/converter.py", line 294, in main() └ <function main at 0x7f2bfcd0c7a0>

File "tools/converter.py", line 219, in main mod = torch.jit.trace(net, x, check_trace=False) │ │ │ │ └ tensor([[[[0.6972, 0.8814, 0.4020, ..., 0.3840, 0.8883, 0.2880], │ │ │ │ [0.5202, 0.6907, 0.6853, ..., 0.8505, 0.1018, 0.... │ │ │ └ Detector( │ │ │ (backbone): TinyNAS( │ │ │ (block_list): ModuleList( │ │ │ (0): Focus( │ │ │ (conv): ConvBNAct( │ │ │ (conv):... │ │ └ <function trace at 0x7f2c138e3050> │ └ <module 'torch.jit' from '/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/jit/init.py'> └ <module 'torch' from '/home/bright/anaconda3/envs/DAMO-YOLO/lib/python3.7/site-packages/torch/init.py'>

RuntimeError: Tracer cannot infer type of [BoxList(num_boxes=0, image_width=640, image_height=640, mode=xyxy)] :Could not infer type of list element: Only tensors and (possibly nested) tuples of tensors, lists, or dictsare supported as inputs or outputs of traced functions, but instead got value of type BoxList.

tinyvision / DAMO-YOLO

Export the model to Torch Script #101