aimagelab / VATr

MIT License
72 stars 4 forks source link

result issue @ fid. #11

Open Fyzjym opened 1 year ago

Fyzjym commented 1 year ago

Hi, I'm once again amazed by your great work. However, there seem to be some issues with testing the FID in the four scenarios. According to GANwriting's description,

My understanding is as follows:

Two dictionaries: in-vocabulary dictionary (114 words) and out-of-vocabulary dictionary (400 words). Five hundred authors: visible style (339, train) and invisible style (161, test). Synthetic dataset: a total of 1000 samples, 339 * 2 + 161 + 2. However, the results I obtained differ significantly from the levels reported by VATr (delta over 50). Is my understanding correct, or is there something I'm missing?

Looking forward to your reply! 😄

Fyzjym commented 1 year ago

or ,calculated the FID for each of the four scenarios according to the code provided by GANwriting, without the step of "calculating separately and then averaging."