search

dk-liang / CLTR

[ECCV 2022] An End-to-End Transformer Model for Crowd Localization
MIT License
88 stars 13 forks source link

Reproduce NWPU testset issue #19

Closed EDDChang closed 1 year ago

EDDChang commented 1 year ago

Hi, I would like to know how to reproduce the performance in NWPU testset on the ranking website https://www.crowdbenchmark.com/index.html There are two questions.

  1. How to preprocess the images? There is no processed test image in the files from your download link. By observing image size of dir images and images_2048. I guess the resize rule is let the width of image no more than 2048 otherwise reduce the length and width proportionally. But I got O_MAE more than 200 when using offered weights or the model train by myself(follow your instructions in README).

  2. Should I use same code except the section dealing with ground-truth in dataset.py for test images?

dk-liang commented 1 year ago

We use the original image without resizing for testing

EDDChang commented 1 year ago

Could you please provide the inference code? I used the checkpoint you offered and inference without resizing and got O_MAE = 105 on NWPU test. Thanks!

dk-liang commented 1 year ago

The provided NWPU pre-trained weight may not be the best for the test set. Actually, we find that the nwpu-test set need to submit much different epoch checkpoints to obtain the best results, e.g., although the Val set only achieves 65 MAE, the test set may achieve the best result. In contrast, when the Val set achieves the best results, the test set may not be the best.