Failed to reproduce the eval results of GPT Generation

zh460045050 / VQGAN-LC

62 stars 6 forks source link

Failed to reproduce the eval results of GPT Generation #7

Open ChangyaoTian opened 1 week ago

ChangyaoTian commented 1 week ago

Hey, when I directly run the eval scripts of GPT here with the provided pre-trained ckpts here, I can only get 24.6 FID Score rather than the claimed 15.4. Is something wrong here?

zh460045050 commented 1 week ago

Thank you for reporting the issues. We have rechecked the evaluation scripts and believe the issue may be due to the setting of top-k sampling. It should be set to 10,000 (k=10% of the vocabulary size) rather than 100,000 (without top-k sampling). We have made the necessary modifications.