Problems with model ensembling

[joonjeon]

Hi! I am trying to ensemble two YOLOv9-E models but error occurs as follows:

YOLO 🚀 v0.1-60-g4cf2226 Python-3.9.18 torch-2.2.0+cu118 CUDA:0 (NVIDIA RTX A6000, 48677MiB)

Fusing layers... 
yolov9-e summary: 839 layers, 68547814 parameters, 0 gradients, 240.7 GFLOPs
Fusing layers... 
yolov9-e summary: 839 layers, 68547814 parameters, 0 gradients, 240.7 GFLOPs
Ensemble created with ['yolov9-e_weights1.pt', 'yolov9-e_weights2.pt']

Traceback (most recent call last):
  File "/home/joonjeon/my_workspace/yolov9/val_dual.py", line 152, in run
    model.warmup(imgsz=(1 if pt else batch_size, 3, imgsz, imgsz))  # warmup
  File "/home/joonjeon/my_workspace/yolov9/models/common.py", line 936, in warmup
    self.forward(im)  # warmup
  File "/home/joonjeon/my_workspace/yolov9/models/common.py", line 854, in forward
    y = self.model(im, augment=augment, visualize=visualize) if augment or visualize else self.model(im)
  File "/home/joonjeon/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/joonjeon/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/joonjeon/my_workspace/yolov9/models/experimental.py", line 65, in forward
    y = torch.cat(y, 1)  # nms ensemble
TypeError: expected Tensor as element 0 in argument 0, but got list

Any ideas why this is happening?

[joonjeon]

In the meantime, while the problem above seems to be caused by YOLOv9 having dual detection heads (as far as I understood),

model ensembling doesn't seem to work on single-head models (e.g. yolov9-e-converted.pt), either:

Traceback (most recent call last):
  File "/home/joonjeon/my_workspace/yolov9/val.py", line 389, in <module>
    main(opt)
  File "/home/joonjeon/my_workspace/yolov9/val.py", line 362, in main
    run(**vars(opt))
  File "/home/vsw1/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/joonjeon/my_workspace/yolov9/val.py", line 244, in run
    save_one_json(predn, jdict, path, class_map)  # append to COCO-JSON dictionary
  File "/home/joonjeon/my_workspace/yolov9/val.py", line 46, in save_one_json
    'category_id': class_map[int(p[5])],
IndexError: list index out of range

Please let me know the cause ASAP.

[ytl0623]

For the second question, maybe you can try to modify the copy_paste parameter in hyp.scratch-high.yaml to 0.0.

I solved the same problem by this method.