search

JackEasson / SLPNet_pytorch

SLPNet: Towards End-to-End Car License Plates Detection and Recognition Using Lightweight CNN
56 stars 13 forks source link

Possible leakage in validation accuracy #9

Closed cenkbircanoglu closed 3 years ago

cenkbircanoglu commented 3 years ago

https://github.com/JackEasson/SLPNet_pytorch/blob/affbf95fe449c5ff82a596ebd087cfbf66cc47aa/train.py#L366

Using this line in the evaluation step results in the wrong accuracy value for the validation dataset. Is it a bug or am I missing some part?

JackEasson commented 3 years ago

What's the wrong accuracy value? 0 or others? And I want to know in which epoch you met this problem. I didn't meet that before.

cenkbircanoglu commented 3 years ago

Let me change my question to this: Is it fair to use online_distribute_ctc_targets function in validation phase?

JackEasson commented 3 years ago

Yes, and you can find more details in "eval.py" .

cenkbircanoglu commented 3 years ago

Maybe I don't understand but to calculate the accuracy there are two points which I think they must be different.

  1. By using online_distribute_ctc_targets you use the label information to predict,
  2. This for i, pred_label in enumerate(pred_labels): line ignores the images if there is no plate found by the model.

Can you please help me to understand these two parts?

JackEasson commented 3 years ago

In online_distribute_ctc_targets, I use predicted bboxes and ground truth to calculate a IOU-like measure, which will be used for selecting the bboxes that really detect license plates (with a threshold 0.5 or 0.7, you can see it as IoU). After that, the reserved 'pred_labels' is the reserved bboxes' numbers. So the accuracy is just for recognition parts. Only the bbox has detected a lp, it's number recognitioned will be used to calculate accuracy with it's number label.