Metrics do not match paper

[Bit0r]

Hello, I used the code for this project to conduct experiments, but I found the metrics of the results do not match what is described in the paper.

Here are the metrics from the paper:

And here are the results I got when running the code for this project:

As you can see, the metrics for each class are lower than what is reported in the paper. The mAP0.5 is even 10% lower!

Could you provide the pretrained weights and training results you have? I'd like to compare with the experiments we have conducted to see if there are any differences in the details.

[hexiao0275]

Hi, sorry for the confusion, there may be a problem with the training parameters, try modifying them to fix it. This detector definitely works better than yolov5 on top of the csv file I have put for training. I've been busy lately and will retrain the code on github to check it.

[Annzstbl]

Hello, thank you for your great contribution and code. Using the code and default config file you posted, I got results which were close to @Bit0r 's.

I tried using yolov5l.pt and yolov5l_fusion_transformerx3_hsi_conv.yaml as pretrained weights and config file. And the result shows that mAP0.5is about 0.85, the mAP@.5:.95 is about 0.50, which are still lower than reported in the paper.

Have you retrained the code on github? Hope you can update the latests training parameters.

[hexiao0275]

The final code is the result obtained from yolov5l_fusion_transformerx3_hsi.yaml, which I've updated in train.py, and also need to adjust the learning rate accordingly, so you can try running it. I'll re-run the code and put it on Github before December.