nan value & gt gt2 & KID problems when eval_unconstrained_humanact12

Hi!

When I ran the eval part, I met a few problems about unconstrained, and hope you can help me.

The process is carried out, and no error is reported to stop the program.

The first problem is that there is no KID value, which can refer to the screenshot from the evaluation_results_iter450000_samp1000_scale1_a2m.yaml file that is the evaluated result for the best model.

In the .yaml file, there are many nan values, including accuracy_gen, accuracy_gt, accuracy_gt2, fid_unconstrained, kid_unconstrained, multimodality_gen, multimodality_gt, multimodality_gt2, precision_unconstrained, and recall_unconstrained.

To double-check the problem, I also evaluated the best model given by the official zip, i.e. ./save/unconstrained/model000450000.pt But the problems still exist. Screen Shot 2024-02-19 at 18 11 17

Finally, I want to know why we need gt and gt2, which means two ground truths.

Thanks in advance, and looking forward to your early reply~

evaluation_results_iter450000_samp1000_scale1_a2m.yaml file attached here

feats:
  accuracy_gen:
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  accuracy_gt:
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  accuracy_gt2:
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  diversity_gen:
  - '6.89782'
  - '6.89011'
  - '6.73952'
  - '6.74987'
  - '6.82585'
  - '6.6621'
  - '6.82288'
  - '6.75882'
  - '6.77904'
  - '6.92874'
  - '6.8772'
  - '6.67161'
  - '6.83096'
  - '6.77628'
  - '6.77839'
  - '6.62971'
  - '7.01509'
  - '6.90686'
  - '6.81357'
  - '6.58429'
  diversity_gen_unconstrained: 17.008365631103516
  diversity_gt:
  - '6.61153'
  - '7.01676'
  - '6.99224'
  - '6.80892'
  - '6.91731'
  - '6.85943'
  - '6.88625'
  - '6.94846'
  - '7.07225'
  - '6.86249'
  - '6.76709'
  - '7.02609'
  - '6.87979'
  - '6.87578'
  - '6.56889'
  - '6.93818'
  - '6.68828'
  - '6.88319'
  - '6.92963'
  - '6.8524'
  diversity_gt2:
  - '6.74916'
  - '6.95423'
  - '6.71648'
  - '6.77055'
  - '6.91332'
  - '6.54734'
  - '6.62869'
  - '6.84382'
  - '6.66525'
  - '6.68556'
  - '6.70404'
  - '6.99287'
  - '6.78967'
  - '6.69709'
  - '6.98212'
  - '6.87764'
  - '6.74566'
  - '6.93538'
  - '6.85223'
  - '6.6096'
  diversity_gt_unconstrained: 20.708080291748047
  fid_gen:
  - '0.297286'
  - '0.259686'
  - '0.422459'
  - '0.228987'
  - '0.320257'
  - '0.30219'
  - '0.284305'
  - '0.314953'
  - '0.262808'
  - '0.298342'
  - '0.217695'
  - '0.361196'
  - '0.412756'
  - '0.210205'
  - '0.232225'
  - '0.309696'
  - '0.294808'
  - '0.299409'
  - '0.277911'
  - '0.237049'
  fid_gt:
  - '-2.84217e-14'
  - '-7.10543e-15'
  - '3.55271e-14'
  - '-6.39488e-14'
  - '-7.81597e-14'
  - '-3.55271e-14'
  - '-9.9476e-14'
  - '5.68434e-14'
  - '-2.13163e-14'
  - '-4.9738e-14'
  - '7.10543e-15'
  - '-2.84217e-14'
  - '-2.13163e-14'
  - '-2.13163e-14'
  - '2.84217e-14'
  - '-7.81597e-14'
  - '-9.9476e-14'
  - '-7.81597e-14'
  - '-9.23706e-14'
  - '7.10543e-15'
  fid_gt2:
  - '0.0515395'
  - '0.0658431'
  - '0.0506913'
  - '0.0591474'
  - '0.0492066'
  - '0.0688688'
  - '0.0487507'
  - '0.0502104'
  - '0.0412811'
  - '0.0510524'
  - '0.0544728'
  - '0.060396'
  - '0.0428596'
  - '0.0442645'
  - '0.0322802'
  - '0.04161'
  - '0.0486726'
  - '0.0648593'
  - '0.0743409'
  - '0.0594989'
  fid_unconstrained: !!python/object/apply:numpy.core.multiarray.scalar
  - &id001 !!python/object/apply:numpy.dtype
    args:
    - f8
    - false
    - true
    state: !!python/tuple
    - 3
    - <
    - null
    - null
    - null
    - -1
    - -1
    - 0
  - !!binary |
    iLdXr9vzQEA=
  kid_unconstrained: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    FIX+FTG/2T8=
  multimodality_gen:
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  multimodality_gt:
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  multimodality_gt2:
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  - nan
  precision_unconstrained: null
  recall_unconstrained: null

GuyTevet / motion-diffusion-model

nan value & gt gt2 & KID problems when eval_unconstrained_humanact12 #191