search

abduallahmohamed / Social-STGCNN

Code for "Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction" CVPR 2020
MIT License
483 stars 141 forks source link

can't reproduce the test results using your models #50

Closed Xiejc97 closed 2 years ago

Xiejc97 commented 3 years ago

Hi,

Thanks for your nice work. I use the models in the checkpoint folder for testing, and run the test.py. But the accuracy is different from the accuracy shown in your paper. So I want to ask the reason.

`** Number of samples: 20

Model being tested are: ['./checkpoint1/social-stgcnn-eth', './checkpoint1/social-stgcnn-hotel', './checkpoint1/social-stgcnn-univ', './checkpoint1/social-stgcnn-zara1', './checkpoint1/social-stgcnn-zara2']

Evaluating model: ./checkpoint1/social-stgcnn-eth Stats: {'min_val_epoch': 248, 'min_val_loss': -0.015072189948775551} Processing Data ..... 100%|██████████| 70/70 [00:01<00:00, 43.19it/s] Testing .... ADE: 0.730797000639612 FDE: 1.2210648458100126

Evaluating model: ./checkpoint1/social-stgcnn-hotel Stats: {'min_val_epoch': 234, 'min_val_loss': -0.014858260246866567} Processing Data ..... 100%|██████████| 301/301 [00:07<00:00, 38.47it/s] Testing .... ADE: 0.4129764052146676 FDE: 0.6802780812341801

Evaluating model: ./checkpoint1/social-stgcnn-univ Stats: {'min_val_epoch': 153, 'min_val_loss': -0.009756729709652235} 0%| | 0/947 [00:00<?, ?it/s]Processing Data ..... 100%|██████████| 947/947 [04:49<00:00, 3.27it/s] Testing .... ADE: 0.4877151096340023 FDE: 0.9114607573058071

Evaluating model: ./checkpoint1/social-stgcnn-zara1 Stats: {'min_val_epoch': 196, 'min_val_loss': -0.01428595929106405} Processing Data ..... 100%|██████████| 602/602 [00:17<00:00, 34.94it/s] Testing .... ADE: 0.33245151309488535 FDE: 0.5195364921152382

Evaluating model: ./checkpoint1/social-stgcnn-zara2 Stats: {'min_val_epoch': 243, 'min_val_loss': -0.013492159500807345} Processing Data ..... 100%|██████████| 921/921 [00:39<00:00, 23.45it/s] Testing .... ADE: 0.3028199381741592 FDE: 0.47966154597607014

Avg ADE: 0.45335199335146525 Avg FDE: 0.7624003444882617`

The ETH dataset's result is 0.73/1.22, while your paper is 0.64/1.11, so the ade is larger. The univ dataset's result is 0.48/0.91, while your paper is 0.44/0.79, so the fde is larger. The results of these two data sets are quite different from your original paper. Can you tell me the specific reasons?

Best, Jincan

abduallahmohamed commented 3 years ago

Hi,

Please refer to: https://github.com/abduallahmohamed/Social-STGCNN/issues/16 https://github.com/abduallahmohamed/Social-STGCNN/issues/26 https://github.com/abduallahmohamed/Social-STGCNN/issues/34 << Exact CVPR settings

Best