search

alexzhou907 / DDBM

163 stars 16 forks source link

FID scores not the same as reported in the paper (using the published E2H model weights) #12

Open limsanky opened 3 weeks ago

limsanky commented 3 weeks ago

Hello @alexzhou907 !

I used the official model weights posted here on the E2H dataset. I followed the code and sampled for 40 steps, but when I evaluate the FID and IS metrics, the results are vastly different compared to the numbers reported on the paper.

With 40 steps (so 119 NFEs), the FID is about 12 and IS is about 4.1. In the paper, the numbers reported are 1.83 FID and 3.73 IS.

Is the published weights in the paper the correct one? Could you please check this out or help me out with this issue?

Best regards

wangya22 commented 2 weeks ago

I can match the results in E2H dataset, but not in DIODE dataset.

With 40 steps (119 NFEs), the FID is 5.4 and IS is 5.87. In the paper, the numbers reported are 4.43 FID and 6.21 IS.

Best regards.