search

SiddhantKapil / LA-Transformer

MIT License
51 stars 12 forks source link

There is something wrong at Evaluation! #8

Open STARK1234 opened 3 years ago

STARK1234 commented 3 years ago

Your test code is very different from others.

  1. AP should be computed at all gallery images, not just 10 samples.
  2. Person re-identification is simple at the same camera, so you should remove gallery images that have the same person-ID and camera-ID with query.

If you ignore these points, the mAP score would significantly get bigger. It's not fair to compare with other works.

The standard eval process: https://github.com/JDAI-CV/fast-reid/blob/43da387b77fec86248154cb00bd42c106e6bf94a/fastreid/evaluation/rank.py

Chenfeng1271 commented 2 years ago

Have you revised his evaluation code? How many of the final results are?

leeeizhang commented 2 years ago

wrong evaluation method!