Different result between models (stage 2) trained from scratch and provided checkpoint

[hlinhn]

Thank you for your work! I ran into some problems training the model (so far only stage 2). The provided checkpoint produced results close to the paper while running eval_gta_stats on GTA test set

path_err,57.38540468983287,94.38535354791149,118.40502109345802,152.13748948140577,87.92836402906703
joint_err,48.21895089142483,66.53363690807149,78.73921221957877,89.39613546094587,61.86808230991969
all_joint_err,73.7825790174371,111.85523976853166,136.15931872049035,167.86036358550845,103.99606321022947

but a model trained from scratch (stage 2 only) with GTA-IM data performed much worse

path_err,116.86713110841265,231.33667114892663,342.41806149832314,474.7360402887518,237.56161213853375
joint_err,172.96141145690794,256.80306212992963,337.8603548423286,437.9991502705907,253.4729992268008
all_joint_err,208.02058091611107,346.54184607816234,479.84686729845066,638.2465369540575,346.61500541655346

This is from the config file provided, with the number of epochs set to 50, the same as the paper. I have also tried

• increasing the number of epochs to 100
• changing the config file to match the numbers in the paper (specifically learning rate, number of DCT components, lambdas) but they always performed around the same.

What am I missing? In case it is a data problem: I downloaded GTA-IM FPS-30 dataset, processed it with process_gta_dataset and got 329 files in data_v2_downsample0.02 folder.

Thank you for your help!