search

WongKinYiu / yolor

implementation of paper - You Only Learn One Representation: Unified Network for Multiple Tasks (https://arxiv.org/abs/2105.04206)
GNU General Public License v3.0
1.99k stars 518 forks source link

How do I use YOLOR-SSSS-DWT model? #9

Open JHLEE17 opened 3 years ago

JHLEE17 commented 3 years ago

I couldn't find YOLOR(or YOLOv4)-SSS(or SSSS)-DWT(or S2D) model in this repo. It also has no detailed information in the paper.

Q1. How do I use DWT model or DWT module?

Q2. Can you provide DWT model and the weights?

Q3. What is S2D in YOLOR-SSSS and what is the difference between S2D and DWT? (not essential)

WongKinYiu commented 3 years ago

Q1. 

# parameters
nc: 80  # number of classes
depth_multiple: 0.33  # model depth multiple
width_multiple: 0.50  # layer channel multiple

# anchors
anchors:
  - [12,16, 19,36, 40,28]  # P3/8
  - [36,75, 76,55, 72,146]  # P4/16
  - [142,110, 192,243, 459,401]  # P5/32

# yolov4s backbone
backbone:
  # [from, number, module, args]
  [[-1, 1, DWT, []],  # 0
   [-1, 1, DWT, []],  # 1
   [-1, 1, DWT, []],  # 2
   [-1, 8, BottleneckCSP, [256]],
   [-1, 1, Conv, [512, 3, 2]],  # 4-P4/16
   [-1, 8, BottleneckCSP, [512]],
   [-1, 1, Conv, [1024, 3, 2]], # 6-P5/32
   [-1, 4, BottleneckCSP, [1024]],  # 7
  ]

# yolov4s head
# na = len(anchors[0])
head:
  [[-1, 1, SPPCSP, [512]], # 8
   [-1, 1, Conv, [256, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [5, 1, Conv, [256, 1, 1]], # route backbone P4
   [[-1, -2], 1, Concat, [1]],
   [-1, 2, BottleneckCSP2, [256]], # 13
   [-1, 1, Conv, [128, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [3, 1, Conv, [128, 1, 1]], # route backbone P3
   [[-1, -2], 1, Concat, [1]],
   [-1, 2, BottleneckCSP2, [128]], # 18
   [-1, 1, Conv, [256, 3, 1]],
   [-2, 1, Conv, [256, 3, 2]],
   [[-1, 13], 1, Concat, [1]],  # cat
   [-1, 2, BottleneckCSP2, [256]], # 22
   [-1, 1, Conv, [512, 3, 1]],
   [-2, 1, Conv, [512, 3, 2]],
   [[-1, 8], 1, Concat, [1]],  # cat
   [-1, 2, BottleneckCSP2, [512]], # 26
   [-1, 1, Conv, [1024, 3, 1]],

   [[19,23,27], 1, IDetect, [nc, anchors]],   # Detect(P3, P4, P5)
  ]

Q2. yolor-ssss-dwt.pt

Q3. (a) discrete wavelet transform -- DWT, (b) spatial to depth -- S2D image

JHLEE17 commented 3 years ago

Thank you for your kind reply. @WongKinYiu

JHLEE17 commented 3 years ago

And if i want to use YOLOR-SSS-DWT model, do i just replace the #DWT 2 with #Stage B2 ??


#parameters
nc: 80  # number of classes
depth_multiple: 0.33  # model depth multiple
width_multiple: 0.50  # layer channel multiple

# anchors
anchors:
  - [12,16, 19,36, 40,28]  # P3/8
  - [36,75, 76,55, 72,146]  # P4/16
  - [142,110, 192,243, 459,401]  # P5/32

# yolov4s backbone
backbone:
  # [from, number, module, args]
  [[-1, 1, DWT, []],  # 0
   [-1, 1, DWT, []],  # 1

   # [-1, 1, DWT, []],  # DWT 2

   [-1, 8, BottleneckCSP, [128]],   
   [-1, 1, Conv, [256, 3, 2]],  # Stage B2

   [-1, 8, BottleneckCSP, [256]],
   [-1, 1, Conv, [512, 3, 2]],  # 4-P4/16
   [-1, 8, BottleneckCSP, [512]],
   [-1, 1, Conv, [1024, 3, 2]], # 6-P5/32
   [-1, 4, BottleneckCSP, [1024]],  # 7
  ]

# yolov4s head
# na = len(anchors[0])
head:
  [[-1, 1, SPPCSP, [512]], # 8
   [-1, 1, Conv, [256, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [5, 1, Conv, [256, 1, 1]], # route backbone P4
   [[-1, -2], 1, Concat, [1]],
   [-1, 2, BottleneckCSP2, [256]], # 13
   [-1, 1, Conv, [128, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [3, 1, Conv, [128, 1, 1]], # route backbone P3
   [[-1, -2], 1, Concat, [1]],
   [-1, 2, BottleneckCSP2, [128]], # 18
   [-1, 1, Conv, [256, 3, 1]],
   [-2, 1, Conv, [256, 3, 2]],
   [[-1, 13], 1, Concat, [1]],  # cat
   [-1, 2, BottleneckCSP2, [256]], # 22
   [-1, 1, Conv, [512, 3, 1]],
   [-2, 1, Conv, [512, 3, 2]],
   [[-1, 8], 1, Concat, [1]],  # cat
   [-1, 2, BottleneckCSP2, [512]], # 26
   [-1, 1, Conv, [1024, 3, 1]],

   [[19,23,27], 1, IDetect, [nc, anchors]],   # Detect(P3, P4, P5)
  ]
WongKinYiu commented 3 years ago 
# parameters
nc: 80  # number of classes
depth_multiple: 0.33  # model depth multiple
width_multiple: 0.50  # layer channel multiple

# anchors
anchors:
  - [12,16, 19,36, 40,28]  # P3/8
  - [36,75, 76,55, 72,146]  # P4/16
  - [142,110, 192,243, 459,401]  # P5/32

# yolov4s backbone
backbone:
  # [from, number, module, args]
  [[-1, 1, DWT, []],  # 0
   [-1, 1, DWT, []],  # 1
   [-1, 2, BottleneckCSP, [128]],
   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
   [-1, 8, BottleneckCSP, [256]],
   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
   [-1, 8, BottleneckCSP, [512]],
   [-1, 1, Conv, [1024, 3, 2]], # 7-P5/32
   [-1, 4, BottleneckCSP, [1024]],  # 8
  ]

# yolov4s head
# na = len(anchors[0])
head:
  [[-1, 1, SPPCSP, [512]], # 9
   [-1, 1, Conv, [256, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [6, 1, Conv, [256, 1, 1]], # route backbone P4
   [[-1, -2], 1, Concat, [1]],
   [-1, 2, BottleneckCSP2, [256]], # 14 
   [-1, 1, Conv, [128, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [4, 1, Conv, [128, 1, 1]], # route backbone P3
   [[-1, -2], 1, Concat, [1]],
   [-1, 2, BottleneckCSP2, [128]], # 19
   [-1, 1, Conv, [256, 3, 1]],
   [-2, 1, Conv, [256, 3, 2]],
   [[-1, 14], 1, Concat, [1]],  # cat
   [-1, 2, BottleneckCSP2, [256]], # 23
   [-1, 1, Conv, [512, 3, 1]],
   [-2, 1, Conv, [512, 3, 2]],
   [[-1, 9], 1, Concat, [1]],  # cat
   [-1, 2, BottleneckCSP2, [512]], # 27
   [-1, 1, Conv, [1024, 3, 1]],

   [[20,24,28], 1, IDetect, [nc, anchors]],   # Detect(P3, P4, P5)
  ]
JHLEE17 commented 3 years ago

I'm sorry to keep bothering you. (Q1) How did you implement YOLOR-S,SS and when will the code be released?

(Q2)The models on the paper branch are written as yaml files, how can I change them to the default branch models' cfg files? And I wonder how to do the opposite.

rodja commented 3 years ago

I would also be interested in a "yolor-ssss-dwt.cfg" file which works with the pretrained weights which @WongKinYiu has provided in https://github.com/WongKinYiu/yolor/issues/9#issuecomment-846319563.

augmentedstartups commented 3 years ago

Also interested in a detailed tutorial or steps on implementing YOLOR-SSS or SSSS.

WongKinYiu commented 3 years ago

[yolor-s-dwt.pt] [yolor-ss-dwt.pt] [yolor-sss-dwt.pt] [yolor-ssss-dwt.pt] [yolor-sss-s2d.pt] [yolor-ssss-s2d.pt]

to create dwt layer in main branch (cfg style):

[dwt]

to create s2d layer in main branch (cfg style):

[reorg]
daitv73 commented 2 months ago

hi @WongKinYiu the link for those model [yolor-s-dwt.pt] [yolor-ss-dwt.pt] [yolor-sss-dwt.pt] [yolor-ssss-dwt.pt] [yolor-sss-s2d.pt] [yolor-ssss-s2d.pt] can't download for now. could you help us reupload ?