search

JihuaPeng / KTPFormer

65 stars 5 forks source link

Evaluation results #10

Open hs1ang-hsu opened 3 months ago

hs1ang-hsu commented 3 months ago

Thank you for the awesome work! I encountered some problems when running the evaluation following your instructions.

  1. The results are slightly different from yours. The average MPJPE is 40.2 mm for me, while 40.1 mm in the paper. Are there any random processes in the inference, resulting in different results on different devices?

  2. I would like to test the diffusion-based methods in which you used KTPFormer as the denoiser in the D3DP. Is the MPJPE of 33.0 mm in the paper tested with J-Best♯ setting, 20 hypotheses, and 10 iterations?

Again, thank you for the work! I look forward to your reply!

JihuaPeng commented 3 months ago
  1. The model trained and tested on Nvidia 3090 yields the MPJPE of 40.1mm (CPN) and 19.0mm (GT) in the paper, while the MPJPE of 40.2mm (CPN) and 18.9mm (GT) are trained and tested on Nvidia 4090. I infer that the different results may be due to the difference in the devices.
  2. Yes. We follow the same setting J-Best♯ of D3DP.