tusharsangam / TransVisDrone

MIT License
29 stars 3 forks source link

Questions about your results #8

Closed 1430329743 closed 8 months ago

1430329743 commented 8 months ago

Hello, I have checked your results through the paper and your log file in github, and I found a very interesting phenomenon, that is, in the dataset FLD, the strongest result of your MAP is 0.72456 in the epoch, but the result in the verification set and the paper is 0.754. Why is that? In the second point I noticed that the number of images also changed to 19466, but the number of images was not as large if Dogfight comments were used Looking forward to your reply

tusharsangam commented 8 months ago

https://github.com/tusharsangam/TransVisDrone/blob/main/runs/val/Fl/image_size_1280_temporal_YOLO5L_5_frames_FL_end_to_end/best_augment_full_save/results.txt

You can refer to this run which has 0.754 mAP., The number of test images is 19466 According to your 2nd point, there is only a difference of 8 frames from what was reported in Dogfightn which is not a big difference this might have been caused due to faulty labels, splitting of data, or fault in original reporting. Rest assured we have followed the same standards when splitting the data when it came to FL drones & NPS drones for fair comparison.

1430329743 commented 8 months ago

Yes, in FLD, I notice that your val result has 0.754 mAP, but the best result in train is only 0.72456. I wonder why is the result in val better than in train As you mentioned in your paper, the FLD data set is half divided into training and half into testing (val), so in my opinion, the strongest data in train is equal to the best results in val. As sent by https://github.com/tusharsangam/TransVisDrone/blob/main/runs/val/Fl/image_size_1280_temporal_YOLO5L_5_frames_FL_end _to_end/best_augment_full_save/results.txt, Indeed at https://github.com/tusharsangam/TransVisDrone/blob/main/runs/train/FL/image_size_1280_temporal_YOLO5L_5_frames_FL_end The best result in _to_end/results.csv is also epoch19, but the MAP of the two EPOch19s is different. Epoch19-0.72456 in train and epoch-19-0.754 in val are the best results. Is the training log corresponding to this val not this one, or are there other data enhancements or different evaluation metrics used in the results?

1430329743 commented 8 months ago

I want to confirm with you that Table I in your paper is a fair comparison of experiments on NPS and FLD datasets, right? Adopted are Dogfig hthttps://github.com/mwaseema/Drone-Detection/tree/main/annotations data, because the number of comments to again the Dogfig less than originally not accurate annotation of graphics, I would like to ask if only the reannotated comments in Dogfig are needed, or if the two are merged with the original full version without exact comments