search

Cartucho / mAP

mean Average Precision - This code evaluates the performance of your neural net for object recognition.
Apache License 2.0
2.93k stars 909 forks source link

New Features Discussion #1

Open Cartucho opened 6 years ago

Cartucho commented 6 years ago

Calculating AP for each class given the PR (Precision Recall) curve. tvmonitor

Calculating the mAP: map

Cartucho commented 6 years ago

Animation: map2

Cartucho commented 6 years ago

Showing area used to calculate the AP: ap_pottedplant

Cartucho commented 6 years ago

intersection_over_union_-_visual_equation

andikira commented 6 years ago

Many thanks for your work. I think it will be great if we have an option to save images with ground-truth box and prediction box in one folder like this image.

image

Cartucho commented 6 years ago

ok cool, so in the result/ folder (the one created after running the code) we could have a result/images/ folder.

We could also add more info to that image, don't you agree? It would be nice to see the confidence of the prediction for example.

What do you think?

andikira commented 6 years ago

yeah that must be cool, wait, i have no "result" folder after run the code, it will be good if we can generate txt file showing the mAP result. and how to generate curve and diagram above ?

i actually want to contribute in this repo but my programming skill is so noob, sorry for that.

Cartucho commented 6 years ago

For the plots you must install matplotlib:

python -mpip install -U matplotlib

with that, the results folder should show up.

Cartucho commented 6 years ago

new feature: results are now written to a results.txt file

andikira commented 6 years ago

yeah very cool after i install matplotlib i have a curve and diagram in result folder but i got error says "NameError : name 'xrange' is not defined", if this error have a big matter i'll open new issue. txt file with some details like details per image or IoU details will be awesome.

Cartucho commented 6 years ago

yes, please open a new issue, I will fix it asap

I'm working on the images, I will be adding: 1) IoU info 2) confidence info

andikira commented 6 years ago

I have tried new feature, it's awesome. I can save the image file with bounding box and prediction box inside it so i can see the error visually.

another feature :

  1. We have a ground truth diagram showing number of object for each class in all images. How about diagram which show number of object predicted for each class, like showing how much sofa predicted, how much tv predicted etc. ?
  2. and the txt result, how about insert more information such as number of object (ground-truth and prediction) for each class, or even the precision-recall table ?

thank you very much for your work and your fast response

Cartucho commented 6 years ago

I added the IoU and the confidence info, what do you think?

diningtable_prediction3

andikira commented 6 years ago

this is so awesome, can't wait for next feature.

Cartucho commented 6 years ago

New feature added: Plot the number of predicted objects per class!

andikira commented 6 years ago

The new feature is so cool, can we have all the data in diagram or curve in txt files (number of predicted object, number of ground-truth object, precision, recall, etc) ? i just see mAP and AP of each object in txt file.

Cartucho commented 6 years ago

How should it be organized? Like this?

  1. number of ground-truth objects
  2. number of predicted objects
  3. AP and precision/recall for each class
  4. finally, mAP

Btw, should we use .txt or .json? .json would be easier to parse/use from code and smaller.

andikira commented 6 years ago

it's up to you to set the order, i think that order is good enough actually i don't have any experience for .json file, .txt file will be good for me but if you think .json file is better to organize a lot of data, i think that will be nice to use .json.

Cartucho commented 6 years ago

New feature: Added previously discussed info to the file results/results.txt

andikira commented 6 years ago

Great, I'll try it asap,

====edite=====

i have tried new repo, that's very great, this is all i need. All darkflow user have to use this repo for their project. Thank you so much sir.

Cartucho commented 6 years ago

@andikira new feature: Adding values to bars

image

Cartucho commented 6 years ago

And info about true and false predictions:

image

Cartucho commented 6 years ago

I think I will rotate the plots horizontally to make them easier to read!

Cartucho commented 6 years ago

@andikira even better now:

image

image

andikira commented 6 years ago

Waw that's so awesome, actually i want to make graph like that, that's why i request to insert detail in txt. This update will make my life easier 😆

Thank you @Cartucho 👍 (and ready for new error issue from me haha)

JeallyBeans commented 6 years ago

Really liked this repo! May i suggest that you add a yolo conversion for the predicted .txt files aswell ? I got it working by changing 3 lines in the convert_gt_yolo.py code, i can show you if you want :+1:

Cartucho commented 6 years ago

@JeallyBeans great!

Yeah, could you send the file to me? to.cartucho@gmail.com If you prefer you can also commit it in a PR.

Cartucho commented 6 years ago

@JeallyBeans Added! Thank you!

JeallyBeans commented 6 years ago

NP :+1:

97andrea97 commented 3 years ago

Hey, great work. If I am not wrong, each point in the PR curve represents a confidence threshold at which the Recall has changed. I think it would be a good idea to add the value of the confidence threshold in the plot (so that one can choose the optimal threshold). Or to output such info in another way. What do you think?