learning rate

[zazada]

Hi, could you please tell me how to set a different learning rate for each layers? And, how can i modify the cfg file to add another convolutional layer between two nonadjacent layers such as layer5 and layer7, and initialize with random parameters? Thank you!

[AlexeyAB]

@zazada Hi, you can't set a different learning rate for each layers without source code changing.

---

What do you mean about "add another convolutional layer between two nonadjacent layers such as layer5 and layer7"?

• Do you want to add layer before layer6 or after layer6?
• Or do you want to add residual connection (shortcut or route), in a such way as in inception-networks? You can look at examples how to do this:
  • resnet-34 with shortcut: https://github.com/AlexeyAB/darknet/blob/12c1d55942a2d89ab14e131feb54645f9ce6a0e8/cfg/msr_34.cfg
  • densenet201 with route: https://github.com/AlexeyAB/darknet/blob/12c1d55942a2d89ab14e131feb54645f9ce6a0e8/cfg/densenet201.cfg

[zazada]

Thank you very much :)

[MyVanitar]

@AlexeyAB

Do you have any idea why using a deeper network does not improve the detection accuracy that much?

[AlexeyAB]

@VanitarNordic No, I have no ideas. Because theoretically, at least one of the models densenet201_yolo2.cfg or resnet50_yolo.cfg should have better accuracy than yolo-voc.cfg/yolo-voc.2.0.cfg: https://github.com/AlexeyAB/darknet/issues/179#issuecomment-330047738

Maybe this is somehow related to that in Yolo for some outputs used soft-max() * logistic_activation(), for some logistic_activation(), for some exp() * const - in contrast to one the same activation function for all outputs in classification networks.