search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.58k stars 7.94k forks source link

delta Object loss function #185

Open htian01 opened 6 years ago

htian01 commented 6 years ago

in region.c -> forward function, when calculate the delta for obj l.delta[best_index + 4] = l.object_scale (1 - l.output[best_index + 4]) logistic_gradient(l.output[best_index + 4]); while means : (1-x)(1-x)x when x near the 0, the delta is also tend to be 0(while it should be 1) It seems to make my model's P(obj) tends to zeros sometimes. is there anything wrong? longing for your answer.

AlexeyAB commented 6 years ago

For the initial x=0 will be delta = (1-1/(1+exp(0)))*(1-1/(1+exp(0)))*(1/(1+exp(0))) = 0.125

I.e. if at the beginning of the function forward_region_layer() the l.output[best_index + 4] == 0, then at the end of the function l.delta[best_index + 4] = 0.125 if object_scale=1 and rescore=0.

  1. In this case, l.output[best_index + 4] is already earlier logistically activated: https://github.com/AlexeyAB/darknet/blob/c38ec6241985a3302f17b6178128cbe4ea2ce858/src/region_layer.c#L155

So for initiall x == 0 will be l.output[best_index + 4] = 0.5 https://en.wikipedia.org/wiki/Logistic_function

  1. So in this line: https://github.com/AlexeyAB/darknet/blob/c38ec6241985a3302f17b6178128cbe4ea2ce858/src/region_layer.c#L302

Really delta = (1 - logistic(x)) * (1 - logistic(x)) * logistic(x) = (1-1/(1+exp(-x)))*(1-1/(1+exp(-x)))*(1/(1+exp(-x))) if object_scale=1 and rescore=0

Chart: en plot

htian01 commented 6 years ago

thanks, i miss the activating code. While i notice it because i find that in darknet, region_layer.c : forward_region_layer l.delta[obj_index] = l.object_scale * (1 - l.output[obj_index]); however, in your code, l.delta[best_index + 4] = l.object_scale * (1 - l.output[best_index + 4]) * logistic_gradient(l.output[best_index + 4]); i don't understand why times a gradient of that points? Is that in fact the same as the darknet while I miss something again? or you made it deliberate to boost the performance?

Further more, with a benchmark your code might be more convincing. While I'm a undergraduate starting lean cv, your code is more beautiful than darknet ,at least more clear and easy to read, but without mAP, it's frustrated to recommd it to others.

AlexeyAB commented 6 years ago

I didn't change anything significant in the code concerning the work of the neural network. This repo is forked at this commit:

I just optimized the part of the code responsible for loading and displaying video streams, this gives + 30-50% FPS for FullHD (1920 x1080) and more than +300% FPS for UltraHD (4K).

After that commit since 27 Nov 2016 Joseph changed Darknet many times to make it more accurate, for example, he try to keep unchanged aspect ratio when image resized from original size (for example 1920x1080) to the network size (for example 416x416) - this also slowed the speed slightly. The code is still fairly simple and understandable to other developers compared to other frameworks: Caffe, Tensorflow, Torch...

Darknet is purely a R&D project, it is not written so that it conforms to all programming rules and conventions - the code contains a program and logical errors, and can work slowly. But this project written so that it can be repeatedly changed very quickly by one person.

So my repository almost completely corresponds to the description of the neural network Yolo v2 in the original article: https://arxiv.org/abs/1612.08242 And has accuracy:

yolov2