On which DISTS implementation are the test values based on

[CrHasher]

I tested both implementations I found so far on github: https://github.com/dingkeyan93/DISTS/commits/master/DISTS_pytorch/DISTS_pt.py (and) https://github.com/dingkeyan93/IQA-optimization/commits/master/IQA_pytorch/DISTS.py

The first one is IMO the official one, but you based the tests on the second one. All I would like to know is why?

def test_dists_simmilar_to_official_implementation() -> None:
    # Baseline scores from: https://github.com/dingkeyan93/DISTS
    loss = DISTS()

    # Greyscale images
    goldhill = torch.tensor(imread('tests/assets/goldhill.gif'))[None, None, ...] / 255.0
    goldhill_jpeg = torch.tensor(imread('tests/assets/goldhill_jpeg.gif'))[None, None, ...] / 255.0

    loss_value = loss(goldhill_jpeg, goldhill)
    baseline_value = torch.tensor(0.19509) // THIS HERE
    assert torch.isclose(loss_value, baseline_value, atol=1e-3), \
        f'Expected PIQ loss to be equal to original. Got {loss_value} and {baseline_value}'

When I use https://github.com/dingkeyan93/DISTS/commits/master/DISTS_pytorch/DISTS_pt.py I get: 0.1638164520263672 When I use https://github.com/dingkeyan93/IQA-optimization/commits/master/IQA_pytorch/DISTS.py I get: 0.19509172439575195

So which is the correct one and why?

[zakajd]

Sorry for the late response. Both those repositories are from the same author, so I expected results to be similar. I don't remember the reasoning behind using not-official repository, but when averaged across many images our current implementation gives pretty similar results in terms of SRCC / PLCC (see table in README).

I'll leave this issue open so that other's are aware of this difference.

[CrHasher]

Thanks for your response. I know the implementations are from the same author but I also asked the author about differences and I got no reply here -> https://github.com/dingkeyan93/DISTS/issues/11