search

BenjaminFiltjens / MS-GCN

Skeleton-based action segmentation with multi-stage spatial-temporal graph convolutional neural networks (MS-GCN)
32 stars 5 forks source link

How to Reproduce Paper Results? #12

Closed rd20karim closed 5 months ago

rd20karim commented 5 months ago

Hi, I tried to run an evaluation on pku-mmd (with provided features) using the pretrained model of 100 epochs and down sample factor of 3 and a number of joints set to 25. The batch size was 16 (which normally should not impact the results), but got the following results which are very different from those reported in the paper:

python label_eval.py --dataset pku-mmd
pku-mmd
Split 1
Acc: 44.9860
Edit: 34.5036
Precision@0.50: 0.2075
Recall@0.50: 0.1287
F1@0.50: 15.8899
MCC@0.50: -0.5860
Frame MCC: -0.0769

score: 4.099818404988241
f1: 15.889872173058011
f1 sd: 0.0
mcc: -0.0769023536308153
mcc sd: 0.0
Acc: 44.98599416106504

Thanks for your help!

BenjaminFiltjens commented 5 months ago

Hi, did a quick check on a new environment, with the paper hyperparameters all metrics are +- 2% from those reported: num_stages = 4 num_layers_PG = 10 num_layers_RF = 10 num_f_maps = 64 features_dim = 6 bz = 4 lr = 0.0005 num_epochs = 100 dil = [1,2,4,8,16,32,64,128,256,512]

use the full temporal resolution @ 100fps

sample_rate = 1