search

GaParmar / clean-fid

PyTorch - FID calculation with proper image resizing and quantization steps [CVPR 2022]
https://www.cs.cmu.edu/~clean-fid/
MIT License
903 stars 69 forks source link

Error for InceptionV3W ~ is it different from regular InceptionV3? #3

Open silvanmurre opened 3 years ago

silvanmurre commented 3 years ago

I want to cleanly calculate the FID score because I had some differences due to resizing. I appreciate the work you have done to make the process more consistent.

I am running the module on PyTorch 1.9.0, CUDA 11.2, and get the following error when calculating the FID score in the "clean" mode using the "torchscript_inception" model from https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/metrics/inception-2015-12-05.pt:

RuntimeError: MALFORMED INPUT: lanes dont match On the following line: https://github.com/GaParmar/clean-fid/blob/9e1c478e4f1f2e355e71963155ac56bc654fc9f0/cleanfid/features.py#L47

It is a very peculiar error coming from this file: https://github.com/pytorch/pytorch/blob/fce85480b97d8d79e92e11fbcd3c03a25ae446e0/torch/csrc/jit/tensorexpr/types.h#L35

I'm not familiar with TorchScript but maybe it has something to do with the model being composed/uploaded in another Pytorch version. It is probably better to ask in the Pytorch github what causes this.

However, I want to ask if just using the regular InceptionV3 model provided here would give the same FID score / would also be good practice. In short: calculate FID in "clean" mode but use the "pytorch_inception" model

GaParmar commented 3 years ago

Hi,

Could you provide a minimal code block that can reproduce this error? I have tested the clean-fid computation in a conda env with PyTorch 1.9.0 and CUDA 11.2, this should not cause any large issues.

Regards, Gaurav

GaParmar commented 2 years ago

I was able to reproduce this error. I found that downgrading the PyTorch version to 1.7 can resolve this error.

Let me know if this solution works for you.

Regards, Gaurav

ucalyptus2 commented 2 years ago

@GaParmar I run into the same problem with torch 1.9.1

GaParmar commented 2 years ago

This issue should be resolved in the current version of clean-fid. Can you show the error trace that caused this?