search

experiencor / keras-yolo2

Easy training on custom dataset. Various backends (MobileNet and SqueezeNet) supported. A YOLO demo to detect raccoon run entirely in brower is accessible at https://git.io/vF7vI (not on Windows).
MIT License
1.73k stars 785 forks source link

why do we pass ground truth labels as input to the model #374

Open evilc3 opened 5 years ago

evilc3 commented 5 years ago

the comments in the code say that its a hack or something ? so is it included becz. of some issues with keras or the actual yolo network needs it ? if it needs it then why?

robertlugg commented 5 years ago

What are you referring to? Is it these lines:

    input_image = Input(shape=(self.input_size, self.input_size, 3))
    self.true_boxes = Input(shape=(1, 1, 1, max_box_per_image, 4))

Those self.true_boxes are needed during training to compute cost. There is also code in frontend.py in the predict function like this:

    dummy_array = np.zeros((1, 1, 1, 1, self.max_box_per_image, 4))

That is sort of a hack. Its because the network requires all inputs even if they really aren't used during inferencing (prediction). I am not too familiar but I believe the way to avoid this would be to rebuild the network during inferencing but then adjust it not to have this input and loss. That is possible but a lot of work.

rodrigo2019 commented 5 years ago

actually is quite easy to generate the inference model, take a look here

robertlugg commented 5 years ago

Ah, thanks rodrigo. Perhaps we should all move to your repository!

rodrigo2019 commented 5 years ago

I think that merging in this repository should be a better ideia

karthee320 commented 5 years ago

@rodrigo2019 I don't understand the need for self.true_boxes, If we see in the Batch Generator's getitem method, both y_batch itself is getting filled with the ground truth annotations whereas b_batch getting overwritten with last 10 found annotations in the image. It doesn't make sense, I believe we should consider all the ground truth boxes in the loss function, which is calculated in the first half of the custom loss function. But for some reason, the author is replacing the variables with the values calculated from self.true_boxes. Does anyone know the reason behind it?

rodrigo2019 commented 5 years ago

Maybe for him was a better way to process the data, I have a branch which I refactored the loss function, almost of the times I get a better mAP validation with this refactored formula, but some times the old formula wrote by @experiencor get a better results, so I didn't merged it into the master because I need more tests.

Btw the refactored formula wasn't wrote by me, someone showed it to me and I adpted the code to this repo