rafaelpadilla / review_object_detection_metrics

Object Detection Metrics. 14 object detection metrics: mean Average Precision (mAP), Average Recall (AR), Spatio-Temporal Tube Average Precision (STT-AP). This project supports different bounding box formats as in COCO, PASCAL, Imagenet, etc.
Other
1.08k stars 215 forks source link

No file was found for the selected detection format in the annotations directory #63

Closed HnH19 closed 3 years ago

HnH19 commented 3 years ago

I have all my .txt annotations in the directory file, with the correct format (the 1st one in the list), where does this error comes from then ?

rafaelpadilla commented 3 years ago

Hi @HnH19

Please, could you specify what error message you are receiving?

Also, please, could you attach to your message a sample of a detection file and its associated ground-truth file?

Thanks

HnH19 commented 3 years ago

Hi,

The error message if the following :

%% No File was found No file was found for the selected detection format in the annotations directory. %% ERROR

Here are the annotations for a ground truth-detection pair.

By the way, there is nothing wrong with my ground truth upload, it is working. The error occurs with the detection annotations file.

ground_truth.txt detection.txt

Thank you in advance. HnH

maketo97 commented 3 years ago

I meet the same problem as @HnH19. The format for ground truth and predictions is same also.

rafaelpadilla commented 3 years ago

Hi @HnH19,

I tried to replicate your problem, but no error was shown. See in the images below how I did with the files you provided

image

image

image

Please, make sure you are following the steps below:

1) Put all your groundtruths .txt files in the same folder (ex: groundtruths/) 2) As your groundtruth bounding boxes are in relative formats (Yolo (.txt)), the images are required as the width and height of the images are needed to calculate the position of the bounding boxes. Thus, put all your images in the same folder (ex: images/) 3) Create a .txt file, where each line contains the name of your classes. Example: the first line (line 0) must have the name of the objects represented by class_id=0. 4) Put all your detections .txt files in the same folder. (ex: detections\)

Make sure that the detections, images and groundtruth files of the same image are named the same. Example: Bounding box detections of image /images/image_A.jpg are represented by the file /detections/image_A.txt, and their groundtruth bounding boxes are in the file /groundtruths/image_A.txt.

Find in the file below an example I made with the files you provided. As I did not have your image, I created a dummy 100x100 black image. HnH19.zip

Let me know if that works. :)

rafaelpadilla commented 3 years ago

@maketo97 , please, see the answer above and check if you are doing all steps correctly.

Let me know if that works now ;)

tylertroy commented 3 years ago

Hi @rafaelpadilla, I was seeing the same error. However, when I use your provided data in HnH19.zip and follow the above just as you describe the error no longer appears but the output metrics are all 0. Any thoughts?

COCO METRICS:
AP: 0.0
AP50: 0.0
AP75: 0.0
APsmall: nan
APmedium: 0.0
APlarge: nan
AR1: 0.0
AR10: 0.0
AR100: 0.0
ARsmall: nan
ARmedium: 0.0
ARlarge: nan

PASCAL METRIC (AP per class)
person: 0

PASCAL METRIC (mAP)
mAP: 0.0
rafaelpadilla commented 3 years ago

Hi @tylertroy ,

Thank you for your message. I attached the file with a small change, and that's why the results are all 0. See the explanation:

It is because the groundtruth file img_A.txt reports one object of class "person" (0) 0 0.573211 0.407236 0.347754 0.628942

and the detection img_A.txt reports a detection of object class "dog" (1) 1 0.92 0.566021 0.417603 0.544014 0.666667

Even though the IOU is high, the class of the detected object is different than the groundtruth object. So, the results must be indeed 0.

If you change the class of one of the files, you will see that the results will change :). Just change the ground-truth file img_A.txt to: 1 0.573211 0.407236 0.347754 0.628942

and it will work.

Best regards

rafaelpadilla commented 3 years ago

Closing issue as no update was reported

tylertroy commented 3 years ago

Apologies @rafaelpadilla . Worked as intended.