search

photosynthesis-team / piq

Measures and metrics for image2image tasks. PyTorch.
Apache License 2.0
1.41k stars 120 forks source link

The batch_size may influence the result. #281

Closed jinxiqinghuan closed 3 years ago

jinxiqinghuan commented 3 years ago

I found that when I test the same data set, my batch-size is different, which may cause the final index to be different. For example the PSNR.

zakajd commented 3 years ago

Hi @jinxiqinghuan Can you please provide a minimal working example that demonstrates this bug?

zakajd commented 3 years ago

Can you also specify how large is the difference between runs with different batch sizes? It may be possible that due to reduction='mean' flag values are reduced slightly differently for each batch and thus result in non-identical metric values. Try to run PSNR with reduction='none' and verify that individual values are the same.

jinxiqinghuan commented 3 years ago

Hi, @zakajd Thanks for your answer. I have found the problem, it is not caused by piq. Thank you again for your kindness.