I could not reproduce the results as paper presented.

[wangpichao]

I follow the instructions of training and test description, but I could only get top1:89.9% top5:97.0% top10:97.9% mAP:84.3%. Could you kindly please give me some hint to improve?

[guxinqian]

A margin of error of 0.5% is normal. You can attempt to run 'train.py' again or set the '--test_epochs' in 'test-all.py' to [240,230,220,210]. In addition, the version of pytorch may also influence the results.

[longma2019]

What is the version of your PyTorch? I used 1.4.0 and Rank-1 is only 88.2%. Here are the parameters:

arch='ap3dres50', dataset='mars', distance='cosine', eval_step=10, gamma=0.1, gpu='0,2', height=256, lr=0.0003, margin=0.3, max_epoch=240, num_instances=4, resume='', root='./data/reid/', sample_stride=8, save_dir='log-mars-ap3d', seed=1, seq_len=4, start_epoch=0, start_eval=0, stepsize=[60, 120, 180], test_batch=32, train_batch=32, use_cpu=False, weight_decay=0.0005, width=128, workers=4

[guxinqian]

Pytorch 1.0. The test process in 'train.py' only uses 4 frames per video for evaluation. You may not run 'test-all.py'

[longma2019]

88.2% was got by train.py. After I used test-all.py, I got: top1:89.6% top5:96.7% top10:97.8% mAP:84.5%, which is still lower than the performance reported in the paper (top1:90.1%, mAP:85.1%)

[guxinqian]

The error of 0.5% is reasonable due to the characters of re-id datasets. You can attempt to run 'train.py' again or set the '--test_epochs' in 'test-all.py' to [240,230,220,210]. In addition, the version of pytorch may also influence the results.

[longma2019]

Finally, I got the results similar as the paper after using PyTorch 1.0.1 (python 3.7). Here are the parameters and results:

Args:Namespace(arch='ap3dres50', dataset='mars', distance='cosine', gpu='0', height=256, resume='log-mars-ap3d-4', root='./data/reid/', test_epochs=[150, 160, 170, 180, 190, 200, 210, 220, 230, 240], test_frames=32, width=128, workers=4)

150: top1:90.1% top5:96.8% top10:97.8% mAP:85.0% (best val performance when training)

160: top1:89.8% top5:96.9% top10:98.1% mAP:84.8%

170: top1:89.8% top5:96.8% top10:97.8% mAP:84.9%

180: top1:89.9% top5:96.8% top10:97.7% mAP:84.7%

190: top1:90.1% top5:96.8% top10:98.0% mAP:84.9%

200: top1:89.6% top5:96.4% top10:97.6% mAP:84.6%

210: top1:90.4% top5:96.7% top10:97.7% mAP:84.8%

220: top1:89.8% top5:96.7% top10:97.9% mAP:85.0%

230: top1:90.2% top5:96.7% top10:97.7% mAP:85.0%

240: top1:89.7% top5:96.6% top10:97.7% mAP:84.8%

[guxinqian]

ok