search

tusen-ai / SimpleTrack

MIT License
342 stars 64 forks source link

Want to share with you the result on NuScenes validation set here. I have not changed any setting or configuration. Centerpoint input. 2Hz #15

Closed BaiLiping closed 2 years ago

BaiLiping commented 2 years ago
Per-class results: |   -- | -- AMOTA   AMOTP   RECALL  MOTAR   GT      MOTA    MOTP    MT      ML      FAF     TP      FPFN       IDS     FRAG    TID     LGD |   bicycle         0.414   1.161   0.478   0.846   1993    0.400   0.160   39      67      10.0    943     1451041    9       8       1.59    2.02 |   bus             0.721   0.766   0.755   0.957   2112    0.718   0.388   60      13      4.3     1584    68518      10      17      1.40    1.91 |   car             0.637   0.828   0.700   0.935   58317   0.636   0.241   1714    835     44.6    39656   2564       17504   1157    1019    0.97    1.56 |   motorcy         0.524   1.013   0.583   0.907   1977    0.516   0.243   36      25      7.7     1124    104824     29      29      1.69    2.35 |   pedestr         0.688   0.737   0.773   0.873   25423   0.654   0.291   983     292     55.4    19049   2424       5778    596     466     0.63    1.14 |   trailer         0.384   1.314   0.506   0.711   2425    0.352   0.526   44      56      32.5    1201    3471199    25      31      1.62    2.45 |   truck           0.475   1.104   0.554   0.822   9650    0.446   0.323   149     192     25.1    5238    9304300    112     120     1.70    2.67 |     |   Aggregated results: |   AMOTA   0.549 |   AMOTP   0.989 |   RECALL  0.621 |   MOTAR   0.865 |   GT      14556 |   MOTA    0.532 |   MOTP    0.310 |   MT      3025 |   ML      1480 |   FAF     25.7 |   TP      68795 |   FP      6582 |   FN      31164 |   IDS     1938 |   FRAG    1690 |   TID     1.37 |   LGD     2.01 |  
ziqipang commented 2 years ago

@BaiLiping I have updated my inference files, please refer to README for the dropbox links.

I run SimpleTrack and the performance is correct. In your case, maybe you need to check if you use the correct config file --configs/nu_configs/giou.yaml. For example, my script is python tools/main_nuscenes.py --name cp_sanity --config_path configs/nu_configs/giou.yaml --det_name cp --data_folder /mnt/truenas/scratch/ziqi.pang/datasets/nuscenes/validation_2hz --process 150.

However, I think figuring out the errors will be a lot easier with the inference file I just provided. Good luck!

BaiLiping commented 2 years ago

Got it. Thanks a lot!

BaiLiping commented 2 years ago

i think i know why the result is so different now. The inference file is generated using only 1 lidar sweep right? the inference file i used is generated with 10 aggregated lidar sweeps.

ziqipang commented 2 years ago

@BaiLiping It depends on the detection file you use. The strongest center point uses 10 sweeps, which is my inference file. Maybe yours uses 1 sweep?

BaiLiping commented 2 years ago

mine is 10 sweeps as well. I will have another try with the inference file you provided. Thanks a lot.