How to add the SE attention module to Yolov3-spp-ultralytics？

[kuangbo]

❔Question

'Yolov3-spp-ultralytics' is the best, I want to add SE attention base on it, but I have a problem, as follows:

$python train.py --batch-size 8 --cfg cfg/yolov3-spp-attention.cfg' --weights weights/yolov3-spp-ultralytics.pt
Namespace(adam=False, batch_size=8, bucket='', cache_images=False, cfg='cfg/yolov3-spp-attention.cfg', data='data/jinwen.data', device='', epochs=300, evolve=False, freeze_layers=False, img_size=[320, 640], multi_scale=False, name='', nosave=False, notest=False, rect=False, resume=False, single_cls=False, weights='weights/yolov3-spp-ultralytics.pt')
Model Summary: 227 layers, 6.31298e+07 parameters, 6.31298e+07 gradients        
Optimizer groups: 76 .bias, 76 Conv2d.weight, 75 other                          
Traceback (most recent call last):                                              
  File "train.py", line 123, in train                                           
    ckpt['model'] = {k: v for k, v in ckpt['model'].items() if model.state_dict(
)[k].numel() == v.numel()}
  File "train.py", line 123, in <dictcomp>
    ckpt['model'] = {k: v for k, v in ckpt['model'].items() if model.state_dict(
)[k].numel() == v.numel()}
KeyError: 'module_list.75.Conv2d.weight'

After reading #657 , I try python train.py --batch-size 8 --cfg cfg/yolov3-spp-attention.cfg --weights weights/darknet53.conv.74, but success. I want to add SE attention base on 'Yolov3-spp-ultralytics', also pre-weights 'yolov3-spp-ultralytics.pt' Thank in advance.

Additional context

I just added '[se]', nothing else has changed. 'yolov3-spp-attention.cfg' is as follows:

# after backbone
[se]
reduction=16
# before SPP
######################

[convolutional]
batch_normalize=1
filters=512
size=1
stride=1
pad=1
activation=leaky

[convolutional]
batch_normalize=1
size=3
stride=1
pad=1
filters=1024
activation=leaky

[convolutional]
batch_normalize=1
filters=512
size=1
stride=1
pad=1
activation=leaky

### SPP ###
[maxpool]
stride=1
size=5

[route]
layers=-2

[maxpool]
stride=1
size=9

[route]
layers=-4

[maxpool]
stride=1
size=13

[route]
layers=-1,-3,-5,-6

### End SPP ###

[github-actions[bot]]

Hello @kuangbo, thank you for your interest in our work! Ultralytics has open-sourced YOLOv5 at https://github.com/ultralytics/yolov5, featuring faster, lighter and more accurate object detection. YOLOv5 is recommended for all new projects.

To continue with this repo, please visit our Custom Training Tutorial to get started, and see our Google Colab Notebook, Docker Image, and GCP Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom model or data training question, please note that Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:

• Cloud-based AI systems operating on hundreds of HD video streams in realtime.
• Edge AI integrated into custom iOS and Android apps for realtime 30 FPS video inference.
• Custom data training, hyperparameter evolution, and model exportation to any destination.

For more information please visit https://www.ultralytics.com.

[nanhui69]

@kuangbo do you have a try?

[kuangbo]

@kuangbo do you have a try?

No，I just replace yolov3-spp-ultralytics.pt with darknet53.conv.74

[nanhui69]

@kuangbo what performance you acquire ,such per classes map value.....

[kuangbo]

@kuangbo what performance you acquire ,such per classes map value.....

Just add SEattention module after backbone. Such as P R mAP F1 are similar. However, boxes plotted are better. That is to say, Integrity Recognition is good. My work for Character recognition.

[nanhui69]

@kuangbo what performance you acquire ,such per classes map value.....

Just add SEattention module after backbone. Such as P R mAP F1 are similar. However, boxes plotted are better. That is to say, Integrity Recognition is good. My work for Character recognition.

ok，is darknet53.conv.74 better than yolov3-spp-ultralytics.pt？ i dont try it with darknet53.conv.74......

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Seperendity]

❔Question

'Yolov3-spp-ultralytics' is the best, I want to add SE attention base on it, but I have a problem, as follows:

$python train.py --batch-size 8 --cfg cfg/yolov3-spp-attention.cfg' --weights weights/yolov3-spp-ultralytics.pt
Namespace(adam=False, batch_size=8, bucket='', cache_images=False, cfg='cfg/yolov3-spp-attention.cfg', data='data/jinwen.data', device='', epochs=300, evolve=False, freeze_layers=False, img_size=[320, 640], multi_scale=False, name='', nosave=False, notest=False, rect=False, resume=False, single_cls=False, weights='weights/yolov3-spp-ultralytics.pt')
Model Summary: 227 layers, 6.31298e+07 parameters, 6.31298e+07 gradients        
Optimizer groups: 76 .bias, 76 Conv2d.weight, 75 other                          
Traceback (most recent call last):                                              
  File "train.py", line 123, in train                                           
    ckpt['model'] = {k: v for k, v in ckpt['model'].items() if model.state_dict(
)[k].numel() == v.numel()}
  File "train.py", line 123, in <dictcomp>
    ckpt['model'] = {k: v for k, v in ckpt['model'].items() if model.state_dict(
)[k].numel() == v.numel()}
KeyError: 'module_list.75.Conv2d.weight'

After reading #657 , I try python train.py --batch-size 8 --cfg cfg/yolov3-spp-attention.cfg --weights weights/darknet53.conv.74, but success. I want to add SE attention base on 'Yolov3-spp-ultralytics', also pre-weights 'yolov3-spp-ultralytics.pt' Thank in advance.

Additional context

I just added '[se]', nothing else has changed. 'yolov3-spp-attention.cfg' is as follows:

# after backbone
[se]
reduction=16
# before SPP
######################

[convolutional]
batch_normalize=1
filters=512
size=1
stride=1
pad=1
activation=leaky

[convolutional]
batch_normalize=1
size=3
stride=1
pad=1
filters=1024
activation=leaky

[convolutional]
batch_normalize=1
filters=512
size=1
stride=1
pad=1
activation=leaky

### SPP ###
[maxpool]
stride=1
size=5

[route]
layers=-2

[maxpool]
stride=1
size=9

[route]
layers=-4

[maxpool]
stride=1
size=13

[route]
layers=-1,-3,-5,-6

### End SPP ###

        new_dict = {}
        model_dict = model.state_dict()
        for k, v in ckpt['model'].items():
            if k in model_dict and v.shape == model_dict[k].shape:
                new_dict[k] = v
        model_dict.update(new_dict)
        model.load_state_dict(model_dict)

you can modify load model like this，it can run efficiently.

[glenn-jocher]

@Seperendity thank you for sharing your experience! It sounds like you have made some meaningful progress by adding the SE attention module after the backbone. You can indeed observe improved box plots, which is a positive indication for Integrity Recognition.

Regarding your question about whether darknet53.conv.74 is better than yolov3-spp-ultralytics.pt, it's highly recommended to experiment with both to determine which one works best for your specific use case. Thanks for contributing to the YOLOv3 community!