Open guanbin1994 opened 6 years ago
I have not yet added functionality to draw PR curves, nor to output the tested images predictions. It is not hard to add, but I didn't have the time yet.
question Questions and Help
I can get the AP, but i do not know how to draw pr curve ?
And because the dataset i used to train is medical dataset, so I also want to look the detected test images which can show bbox on images, what should i do to output tested images into a folder.
Thanks!
You may refer to maskrcnn_demo.ipynb in demo folder. Here is the code from predictor.py
`def
compute_colors_for_labels(labels):
"""
Simple function that adds fixed colors depending on the class
"""
colors = labels[:, None] * torch.tensor([2 25 - 1, 2 15 - 1, 2 ** 21 - 1])
colors = (colors % 255).numpy().astype("uint8")
return colors
def overlay_boxes(image, predictions):
"""
Adds the predicted boxes on top of the image
Arguments:
image (np.ndarray): an image as returned by OpenCV
predictions (BoxList): the result of the computation by the model.
It should contain the field labels
.
"""
labels = predictions.get_field("labels")
boxes = predictions.bbox
colors = compute_colors_for_labels(labels).tolist()
for box, color in zip(boxes, colors):
box = box.to(torch.int64)
top_left, bottom_right = box[:2].tolist(), box[2:].tolist()
image = cv2.rectangle(
image, tuple(top_left), tuple(bottom_right), tuple(color), 3
)
return image`
question Questions and Help
I can get the AP, but i do not know how to draw pr curve ? And because the dataset i used to train is medical dataset, so I also want to look the detected test images which can show bbox on images, what should i do to output tested images into a folder. Thanks!
You may refer to maskrcnn_demo.ipynb in demo folder. Here is the code from predictor.py
`def
compute_colors_for_labels(labels): """ Simple function that adds fixed colors depending on the class """ colors = labels[:, None] * torch.tensor([2 25 - 1, 2 15 - 1, 2 ** 21 - 1]) colors = (colors % 255).numpy().astype("uint8") return colorsdef overlay_boxes(image, predictions): """ Adds the predicted boxes on top of the image Arguments: image (np.ndarray): an image as returned by OpenCV predictions (BoxList): the result of the computation by the model. It should contain the field
labels
. """ labels = predictions.get_field("labels") boxes = predictions.bboxcolors = compute_colors_for_labels(labels).tolist() for box, color in zip(boxes, colors): box = box.to(torch.int64) top_left, bottom_right = box[:2].tolist(), box[2:].tolist() image = cv2.rectangle( image, tuple(top_left), tuple(bottom_right), tuple(color), 3 ) return image`
hello, my problem i s how to draw the precision-recall curve
@guanbin1994 didn't https://github.com/facebookresearch/maskrcnn-benchmark/issues/353 address your question?
@fmassa issue I have a similar question. From issue #353 I can get the coco_eval
object that has the precision and recall values. However, the shape of these two arrays is different, and it is not clear how to use them to plot a PR curve. I can see the arrays as being of shape T, R, K, A, M
and T, K, A, M
for precision
and recall
, respectively. So I have 101 precision values at each IoU threshold but only one recall value.
How can I plot the PR curve from these?
pycocotools
gives you the eval at IoU thresholds staring at 0.5 up to 0.95 with step sizes of 0.05. The dimensions of the precision array correspond to the following:
T
: len([.5:.05:1.0])
the 10 IoU ThresholdsR
: len([0:0.01:1])
the 100 Recall ThresholdsK
: The number of category ids you haveA
: A=4
which corresponds to the object area rangesM
: max detections So, if you wanted to plot the curves, you could write something like this:
import numpy as np
import matplotlib.pyplot as plt
all_precision = coco_eval.eval['precision']
pr_5 = all_precision[0, :, 0, 0, 2] # data for IoU@0.5
pr_7 = all_precision[4, :, 0, 0, 2] # data for IoU@0.7
pr_9 = all_precision[8, :, 0, 0, 2] # data for IoU@0.9
x = np.arange(0, 1.01, 0.01)
plt.plot(x, pr_5, label='IoU@0.5')
plt.plot(x, pr_7, label='IoU@0.7')
plt.plot(x, pr_9, label='IoU@0.9')
plt.show()
@ayman-saleh Thank you for your answer!
Just for my understanding, is M
the confidence thresholds that is used? in other words, does it means that when M=100
we just consider the objects with confidence score of 100?
Hmmm is it possible to compute the mAP for a certain threshold?
@isalirezag M
is a list, so by default, i believe it is M = [1, 10, 100]
. This list corresponds to the thresholds on max detections per image.
@ayman-saleh sorry that I got here so late, but how would you go about plotting the precision against the recall? Here you are just plotting the precision against an array from 0 to 1.01, but to my knowledge, recall also has different values, which would make the plot different. Following your code, I plotted this in my program and this came out, which looks a bit odd to me compared to the typical precision vs recall curves.
Hi @ayman-saleh how about the recall array?, the dimensions of precision array is 5
while the dimensions of the recall array is 4
. ie. [TxKxAxM]
. How to extract the recall in a shape that can be plot.
Thank you.
@ayman-saleh The code you provided computes precision. Please how about the recall points?
?
Btw, The array of precision returns values full of ones. something probably associated with my dataset. This seem a strange to me. Have you by any chance come across such behaviour?
Hi @ayman-saleh how about the recall array?, the dimensions of precision array is
5
while the dimensions of the recall array is4
. ie.[TxKxAxM]
. How to extract the recall in a shape that can be plot. Thank you.
Instead of parametrizing the arrays by score threshold, I believe they are parametrized by recall threshold. So you can just do as in here and the result will be a precision vs. recall curve.
Hi @ayman-saleh how about the recall array?, the dimensions of precision array is
5
while the dimensions of the recall array is4
. ie.[TxKxAxM]
. How to extract the recall in a shape that can be plot. Thank you.Instead of parametrizing the arrays by score threshold, I believe they are parametrized by recall threshold. So you can just do as in here and the result will be a precision vs. recall curve.
Hi. Do you know how we could parameterize the precision arrays by score threshold instead of by recall threshold? Thanks
❓ Questions and Help
I can get the AP, but i do not know how to draw pr curve ?
And because the dataset i used to train is medical dataset, so I also want to look the detected test images which can show bbox on images, what should i do to output tested images into a folder.
Thanks!