search

Cysu / open-reid

Open source person re-identification library in python
https://cysu.github.io/open-reid/
MIT License
1.34k stars 349 forks source link

Fail to reproduce triplet loss benchmark on Market1501 (too good to be true?) #43

Closed lugiavn closed 6 years ago

lugiavn commented 6 years ago

Hey, I just downloaded the ran the code today:

python examples/triplet_loss.py -d market1501 -a resnet50 --combine-trainval --logs-dir examples/logs/triplet-loss/market1501-resnet50 -b 64

The only change I made is batch size because I have only 1 GPU, however the result right now is quite different from what's reported on the benchmark websites:

Mean AP: 78.6% CMC Scores allshots cuhk03 market1501 top-1 57.0% 78.1% 87.3% top-5 72.5% 97.0% 95.4% top-10 79.8% 98.7% 97.9%

1) Is this result "legit", it looks too good? 2) Is this comparable to others people result, for example this paper: "In Defense of the Triplet Loss for Person Re-Identification" I think the paper says they are using multiple crop, instead of single crop. There's also a difference in train/test split, I heard? 3) It'll be great if you can release the trained models and also the logs from the benchmark website, many don't have the resource to run the training

lugiavn commented 6 years ago

Hm after 100 epochs, the Mean AP is round 90%, compared to state of the art 70%, something wrong must be going on :/

lugiavn commented 6 years ago

Ok so those are the validation result and the test is done only at the end, kind of weird :/ Still interested to hear about the questions 2 & 3 though

Cysu commented 6 years ago

Use —combine-trainval will result in the situation. It’s just for producing the final model. When tuning the hyperparams, we disabled this option.

Cysu commented 6 years ago

  1. Please refer to #16 for the splits on Market and DukeMTMC

  2. Sorry but I currently don’t have trained model on hand...