search

toshas / torch-fidelity

High-fidelity performance metrics for generative models in PyTorch
Other
1.01k stars 66 forks source link

Some question about 'calculate_metrics' #27

Closed C-water closed 3 years ago

C-water commented 3 years ago

Hi,

I have some problems when using 'calculate_metrics' to calculate FID and KID, and I hope to get your help. Q: I only have 200 groups of images (each containing gt.png and out.png), how do I call calculate_metrics? How to change the input in calculate_metrics(input1, input2,kid=True)?

Thank you.

toshas commented 3 years ago

Hi, the questions is whether you need FID/KID at all when your data is clearly paired. FID/KID compare distributions, that is they can give you a "distance" between two unordered sets of samples. You could of course compose the two said sets from all gt and all out, but the resulting value would be hard to interpret. I would suggest to take a look into perceptual similarity metrics, such as LPIPS (which is also implemented in torch_fidelity, just can only be accessed programmatically). Feel free to ask followup questions in this ticket despite it being closed.