Help on: "RuntimeError: stack expects each tensor to be equal size"

[ghost]

Hi,

Thanks for your great work! When trying to run a very simple example (code pasted below):

# -*- coding: utf-8 -*-
"""
Created on Fri Feb  4 13:18:44 2022

@author: User
"""

import torch_fidelity

path1 = r'/home/rsmalbraak/data/TRAIN_3classes_nofolder'
path2 = r'/home/rsmalbraak/data/Generated_3dlayer_decay_nofolder'

metrics_dict = torch_fidelity.calculate_metrics(
    input1=path1, 
    input2=path2, 
    cuda=True, 
    isc=True, 
    fid=True, 
    kid=False, 
    verbose=False,
    ppl_sample_similarity_resize=64,
)
print(metrics_dict)

I get the following error message:

Traceback (most recent call last):
  File "/home/rsmalbraak/FID_IS.py", line 40, in <module>
    metrics_dict = torch_fidelity.calculate_metrics(
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch_fidelity/metrics.py", line 239, in calculate_metrics
    featuresdict_1 = extract_featuresdict_from_input_id_cached(1, feat_extractor, **kwargs)
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch_fidelity/utils.py", line 372, in extract_featuresdict_from_input_id_cached
    featuresdict = fn_recompute()
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch_fidelity/utils.py", line 360, in fn_recompute
    return extract_featuresdict_from_input_id(input_id, feat_extractor, **kwargs)
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch_fidelity/utils.py", line 345, in extract_featuresdict_from_input_id
    featuresdict = get_featuresdict_from_dataset(input, feat_extractor, batch_size, cuda, save_cpu_ram, verbose)
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch_fidelity/utils.py", line 111, in get_featuresdict_from_dataset
    for bid, batch in enumerate(dataloader):
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
    data.reraise()
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch/_utils.py", line 434, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 1.
Original Traceback (most recent call last):
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "/home/rsmalbraak/.local/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 56, in default_collate
    return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [3, 1024, 1024] at entry 0 and [3, 256, 256] at entry 38

As I have used "ppl_sample_similarity_resize", I do not understand why this error occurs. Do you know how I can fix this?

Looking forward to your reply and thank you very much in advance!

[toshas]

Hi there, two points here:

1. ppl_sample_similarity_resize controls only what pertains to PPL metric calculation, and does not have any effect on other metrics like FID.
2. The library expects all images in the folder to be of the same size, so this is technically just a guard to prevent varying input resolution. This size is essentially a part of your evaluation protocol, which should be reported along with metrics results. When you bring all images to the same size, you'll notice that the values of metrics change a lot depending on the chosen resolution.

HTH

[ghost]

Thank you, that makes a lot of sense! I will make bring all images to the same size.