mean_mAP issue - Githubissues

mama110 commented 5 years ago

I'm training VOC2007, afer 1 epoch, an error shows up:

F:\pytorch-yolov3-master-ul\test.py:122: RuntimeWarning: invalid value encountered in double_scalars print('%15s: %-.4f' % (c, AP_accum[i] / AP_accum_count[i])) aeroplane: nan bicycle: nan bird: nan boat: nan bottle: nan bus: nan car: nan cat: nan chair: nan cow: nan diningtable: nan dog: nan horse: nan motorbike: nan person: nan pottedplant: nan sheep: nan sofa: nan train: nan tvmonitor: nan Traceback (most recent call last): File "train.py", line 268, in var=opt.var, File "train.py", line 224, in train img_size=img_size, File "F:\pytorch-yolov3-master-ul\test.py", line 125, in test return mean_mAP, mean_R, mean_P UnboundLocalError: local variable 'mean_mAP' referenced before assignment

What's the problem?

nirbenz commented 5 years ago

This is because mean_mAP isn't initialized and if a snapshot failed to detect any objects (quite normal for the first few evaluation rounds) it will fail. Add this before the train loop.

mean_mAP, mean_R, mean_P = 0.0, 0.0, 0.0

@glenn-jocher I'd PR this but it's a single line :)

mama110 commented 5 years ago

@nirbenz Thanks.

glenn-jocher commented 5 years ago

@nirbenz thanks, I added this line to test.py in the latest commit!

pprp commented 5 years ago

After running python train.py I got the error below:

WARNING Using GPU0 Only. Multi-GPU issue: https://github.com/ultralytics/yolov3/issues/21
Using cuda:0

      Image      Total          P          R        mAP
      Image      Total          P          R        mAP

mAP Per Class:
/media/cieserver/209b4bc6-b4ad-4a37-b5ee-cde3c165103e/dpj/yolov3/test.py:114: RuntimeWarning: invalid value encountered in double_scalars
  print('%15s: %-.4f' % (c, AP_accum[i] / AP_accum_count[i]))
           head: nan
            eye: nan
           nose: nan

I am using the latest commits and I get the error same as @mama110

mean_mAP, mean_R, mean_P = 0.0, 0.0, 0.0

this solution don't work.

glenn-jocher commented 5 years ago

Do you have more than one GPU?

On Sun, 17 Feb 2019 at 03:09, dpj notifications@github.com wrote:

After running python train.py I got the error below:

WARNING Using GPU0 Only. Multi-GPU issue: https://github.com/ultralytics/yolov3/issues/21 Using cuda:0
  Image      Total          P          R        mAP
  Image      Total          P          R        mAP
mAP Per Class: /media/cieserver/209b4bc6-b4ad-4a37-b5ee-cde3c165103e/dpj/yolov3/test.py:114: RuntimeWarning: invalid value encountered in double_scalars print('%15s: %-.4f' % (c, AP_accum[i] / AP_accum_count[i])) head: nan eye: nan nose: nan

I am using the latest version and I get the error same as @mama110 https://github.com/mama110

mean_mAP, mean_R, mean_P = 0.0, 0.0, 0.0

this solution don't work.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/ultralytics/yolov3/issues/50#issuecomment-464408258, or mute the thread https://github.com/notifications/unsubscribe-auth/AZlyGdvXXoxCJ9xVpx7Ih7YHb0tSazECks5vOLndgaJpZM4ZYJM_ .

-- Glenn Jocher Founder and CEO, Ultralytics LLC +1 301-237-6695

pprp commented 5 years ago

Yes , I have two GPU but I used CUDA_VISIBLE_DEVICES=0

pprp commented 5 years ago

After the training, the weight file can detect the target, so it is very likely that the test part calculation is a problem. In addition, I have a question, how to draw pictures below?

pprp commented 5 years ago

I found a problem with my valid file. The path is wrong and I solved it. Thank you very much for your quick response.

glenn-jocher commented 5 years ago

@pprp you can make the plots below using plot_results() in utils.py. You can see more examples here: https://github.com/ultralytics/yolov3/wiki/Single-Image-Training-Example

pprp commented 5 years ago

Thank you for your kind response! 😄

kenrubiooo commented 5 years ago

Hi @mama110, @glenn-jocher,

I also encountered RuntimeWarning: invalid value encountered in double_scalars. I got the APs but the Targets, P, R, mAP, and F1 are 0 and nans. What could be the problem here? Thank you!

Screenshot from 2019-04-25 14-58-52

glenn-jocher commented 5 years ago

@kenrubiooo Hello, thank you for your interest in our work! This is an automated response. Please note that most technical problems are due to:

Your changes to the default repository. If your issue is not reproducible in a fresh git clone version of this repository we can not debug it. Before going further run this code and ensure your issue persists:

sudo rm -rf yolov3  # remove exising repo
git clone https://github.com/ultralytics/yolov3 && cd yolov3 # git clone latest
python3 detect.py  # verify detection
python3 train.py  # verify training (a few batches only)
# CODE TO REPRODUCE YOUR ISSUE HERE

Your custom data. If your issue is not reproducible with COCO data we can not debug it. Visit our Custom Training Tutorial for exact details on how to format your custom data. Examine train_batch0.jpg and test_batch0.jpg for a sanity check of training and testing data.
Your environment. If your issue is not reproducible in a GCP Quickstart Guide VM we can not debug it. Ensure you meet the requirements specified in the README: Unix, MacOS, or Windows with Python >= 3.7, Pytorch >= 1.0, etc.

If none of these apply to you, we suggest you close this issue and raise a new one using the Bug Report template, providing screenshots and minimum viable code to reproduce your issue. Thank you!

kenrubiooo commented 5 years ago

Hi @glenn-jocher

To clarify, I am just experiencing an error in the test.py when trying to determine the mAP.

glenn-jocher commented 5 years ago

@kenrubiooo run this code exactly and paste your terminal output:

sudo rm -rf yolov3  # remove exising repo
git clone https://github.com/ultralytics/yolov3 && cd yolov3 # git clone latest
python3 test.py  # verify test

ultralytics / yolov3

mean_mAP issue #50