search

Cuogeihong / CEASC

The official implementation of CEASC
Apache License 2.0
99 stars 12 forks source link

Question about testing FPS #4

Open gentlezachary opened 1 year ago

gentlezachary commented 1 year ago

Hello,

I'm trying to test the FPS and I'm having trouble getting accurate results. I tried using the benchmark.py script in mmdetection but the results are different from what was presented in the paper.

Here are two results I got.

  1. dynamic_gfl_res18_visdrone.py

image

  1. baseline_gfl_res18_visdrone.py

image

This is your paper results.

image

What is the possible reseason? Could you please explain how to properly test FPS using mmdetection or suggest an alternative method?

Thank you!

Cuogeihong commented 1 year ago

Yes, you can use benchmark.py for testing.

We calculate time between these codes and use torch.cuda.Event to get more accurate results. https://github.com/Cuogeihong/CEASC/blob/b1b84ce894e1037db8754c0823ba9a2260c12322/mmdet/models/detectors/single_stage.py#L101-L107

Actually, reproducing results on different devices can be difficult. Our inference used 2 Intel Xeon E5-2450v4 CPUs and 1 Nvidia RTX 2080ti GPU. So you can try similar settings and make sure no other tasks are running during the test