search

saiboxx / chexray-diffusion

Code for "Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis" @ PAKDD 2023
MIT License
44 stars 8 forks source link

FID Calculation #6

Closed ahmed1996said closed 10 months ago

ahmed1996said commented 10 months ago

Greetings!

I wanted to reproduce the results presented in the paper, can you provide more details on how the FID was calculated?

For example, did you use the Clean-FID process, and was the network an InceptionV3 network? How many samples did you generate (10k?) ?

Thanks in advance! :)

saiboxx commented 10 months ago

Hi @ahmed1996said , Thanks for asking!

The FID was computed using the torchmetrics package across 25k samples. The same goes for KID.

In this case I might also refer to Chambon et al., which experimented with different network backends than InceptionV3, as there are ongoing discussions on the legitimacy of evaluating medical data on a ImageNet pre-trained network.

Cheers, Tobias

ahmed1996said commented 10 months ago

Thanks for the prompt response, @saiboxx!

Understood, thanks for the clarification. I've noticed that Chambon et al. computed the FID on different backends, including a DenseNet pretrained on X-ray images, which may be more fitting.

In your paper, regarding the FID mentioned in section 5, was it computed using the XRV DenseNet or Inception features?

saiboxx commented 10 months ago

I used the implementation in the torchmetrics package, which afaik only offers the standard InceptionV3 backend :+1:

ahmed1996said commented 10 months ago

Got it, totally missed that in your previous response sorry!

Thanks a lot :)