search

photosynthesis-team / piq

Measures and metrics for image2image tasks. PyTorch.
Apache License 2.0
1.32k stars 114 forks source link

NaN values on KADID10k for IS #309

Open snk4tr opened 2 years ago

snk4tr commented 2 years ago

Describe the bug Computation of IS for some images in KADID10k dataset causes NaN results, which is not a valid output.

To Reproduce Steps to reproduce the behavior:

  1. Checkout to feature/extend_bench
  2. Run tests/resutls_benchmark.py with --dataset=kadid10k and --metrics IS
  3. Print out computed metrics' values and observe torch.tensor(nan) as output for some images
  4. Alternatively, finish computation of IS on all DB metrics and observe an error during computation of correlation values

Expected behavior No NaN values are expected.

Screenshots

image

Additional context Other DB metrics such as FID, MSID, KID and GS might me vulnerable.