chengche6230 / ReST

[ICCV 2023] ReST: A Reconfigurable Spatial-Temporal Graph Model for Multi-Camera Multi-Object Tracking
MIT License
130 stars 15 forks source link

Non-replicable Evaluation on PETS09-S2L1 #15

Closed mkg1204 closed 5 days ago

mkg1204 commented 2 months ago

ReST is really a nice work, and I'm very intersting about it. But when I trying to evaluate it on the PETS09-S2L1 dataset using the provided checkpoints in this repository, I got the following results, which is different from the results reported in your paper. The camera number in the metainfo.json is four, so I want to know which four cameras has been used in this experiments?

         IDF1   IDP   IDR   Rcll   Prcn GT MT PT ML FP FN IDs  FM  MOTA  MOTP IDt IDa IDm 
0       67.2% 67.9% 67.2% 100.0% 100.0%  7  7  0  0  0  0  44   0 88.8% 0.005   6  26   0 
1       35.4% 36.4% 35.4% 100.0% 100.0%  7  7  0  0  0  0  76   0 76.6% 0.000   9  50   0 
2       28.0% 39.2% 28.0% 100.0% 100.0%  7  7  0  0  0  0 171   0 63.8% 0.004  90  70   0 
3       32.6% 42.5% 32.6% 100.0% 100.0%  6  6  0  0  0  0 113   0 62.8% 0.000  49  48   1 
OVERALL 40.8% 47.8% 40.8% 100.0% 100.0% 27 27  0  0  0  0 404   0 73.0% 0.002 154 194   1 

Besides, as shown in the metainfo.json, 90% frames are used to train the models on CAMPUS and PETS09-S2L1 datasets, which is different with other works like LMGP. Why those two datasets are splited in this way?

Thanks for your nice work!

chengche6230 commented 2 months ago

Hi,

We used the first four cameras in our experiment and followed the same setting in Wildtrack for consistency. Feel free to run different experiment settings.

Best

jingliang0910 commented 1 month ago

hi,how can i download CAMPUS and PETS09-S2L1 datasets?I find their website can not be opened.