How to fix valid.py problem.

[mrkieumy]

Dear all, I meet the problem and I probably think cause valid.py because:

1. after training 50 epochs, I test mAP by run: (python valid.py with that weight file, and python scripts/my_eval.py), the result is 0 like this (I have 1 class: person).

AP for person = 0.0002 with 2245 views Mean AP = 0.0002 with total 2245 views

           Results:
------------------------------
person          0.000
==============================
 Average        0.000

But I test mAP on DARKNET with that weight file, the result is 43.68% mAP. --> So the problem with either valid.py or my_eval.py, but I continuously test to see which one error:

2. I use darknet to valid that weight file to get the detection result file (comp4_det_test_person.txt), and I use this repo to run only scripts/my_eval.py on comp4_det_test_person.txt (this produced from darknet), the result is nearly the same on darknet: 42.7% mAP.
3. I using another weight file trained on darknet, test mAP on darknet is OK. 49%, but when testing on this repo those weight file it's still zero. --> So the problem with valid.py file and it has some negatives value on the detection result (center x part) in the comp4_det_test_person.txt file. Does anyone know how to fix this problem? I'm trying to fix this. I look at the code of valid and my_eval 5 days, everything looks fine. So I don't know exactly how to solve this problem. It takes me 5 days, but I don't know how to fix it. Any help will be appreciated. Thanks.

[andy-yun]

would you please attach both comp4_det_test_person.txt ? I will check both and source program. Thanks.

[mrkieumy]

@andy-yun Attached are 2 files from pytorch and darknet. comp4_det_test_person_pytorch.txt comp4_det_test_person_darknet.txt Thanks.

[andy-yun]

@mrkieumy would you test the following codes rather than original correct_yolo_boxes in image.py:

def correct_yolo_boxes(boxes, im_w, im_h, net_w, net_h):
    im_w, im_h = float(im_w), float(im_h)
    net_w, net_h = float(net_w), float(net_h)
    if net_w/im_w < net_h/im_h:
        new_w = net_w
        new_h = (im_h * net_w)/im_w
    else:
        new_w = (im_w * net_h)/im_h
        new_h = net_h

    xo, xs = (net_w - new_w)/(2*net_w), net_w/new_w
    yo, ys = (net_h - new_h)/(2*net_h), net_h/new_h
    for i in range(len(boxes)):
        b = boxes[i] 
        b[0] = (b[0] - xo) * xs
        b[1] = (b[1] - yo) * ys
        if b[0] < 0 or b[1] < 0:
           print( im_w, im_h, net_w, net_h, boxes[i], xo, yo, xs, ys)
        b[2] *= xs
        b[3] *= ys
    return

[andy-yun]

@andy-yun Attached are 2 files from pytorch and darknet. comp4_det_test_person_pytorch.txt comp4_det_test_person_darknet.txt Thanks.

I think the detected objects are different in both files. Thus, in my opinion, the problem is not related to valid.py, but the load weights or something related. Plz check load weights.

[mrkieumy]

Thanks @andy-yun I'll look at the load weight and give you the response.

[mrkieumy]

Hi @andy-yun , I checked both weight file and the new your correct_yolo_boxes() function, but it still the same problem. The problem is valid.py file. I checked like the previous comment. Darknet tests normally, your my_eval.py also tests normally on darknet file. But every time through valid.py, that result can not use for evaluation. And the negative value is not cause the new correct_yolo_boxes() function, its negative because the way we calculate x1,x2,y1,y2 from the (cx,cy,w,h), so the negative value only occurs in x1. I set if x1<0: x1=0. so there is no negative value, but the result is still the same, zero all like: load detection file: results/detection_result_person.txt AP for person = 0.0002 with 2245 views TP: 35, FP: 5216 Precision: 0.008583408161881637 Recall: 0.009541506358530074 Mean AP = 0.0002 with total 2245 views

           Results:
------------------------------
person          0.000
==============================
 Average        0.000

Attached are 3 files I ran valid on both darknet and pytorch on the same file weight. Can you look at again? detection_result_person_pytorch_without_negative.txt detection_result_person_pytorch.txt comp4_det_test_person_darknet.txt

Thanks.

[andy-yun]

@mrkieumy Did you test valid.py using darknet weight file (on same environment?)? Which machine do you train/get the weight file by training ? While did you train your model, how much is the correct ratio of that? I have some curiosity about your environment and weight file because there is no problem in other's experiment. Would you check the environment step by step? Sincerely,

[mrkieumy]

@andy-yun Thank you very much for your response. Now it runs normally after I redownload your repo again and the result is the same with darknet. So, I did something wrong in the code. My fault. I'm sorry to waste your time. Thank you very much for your great repo again. Best regards.