search

layer6ai-labs / dgm-eval

Codebase for evaluation of deep generative models as presented in Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models
MIT License
123 stars 9 forks source link

sFDD? #5

Closed dongzhuoyao closed 4 months ago

dongzhuoyao commented 4 months ago

Hi, do you have the implementation of spatial FDD with Dinov2 feature?

JesseCresswell commented 4 months ago

Hi there - all the metrics we implemented are listed in the README, including Spatial FID.

dongzhuoyao commented 4 months ago

Could you kindly point it out? I cannot find spatial FDD in this dir:

https://github.com/layer6ai-labs/dgm-eval/tree/master/dgm_eval/metrics

JesseCresswell commented 4 months ago

This is the description of sFID from our paper, see Appendix B.1:

The spatial FID (sFID) is the FID computed using a representation from the intermediate mixed 6/conv layer of the Inception-V3 network trained for ImageNet1k, rather than the standard (pool3, 2048 dimensional) layer. The sFID relies on the Inception-V3 network, and thus we do not report it for other architectures.

Since sFID is just FID using a different layer of the Inception net, it doesn't make sense to define a DINOv2 version of sFID. We would recommend using FD_DINOv2.