non-exact reproducibility on cpu

[themurtazanazir]

torch.Generator creates different numbers for the same seed on cpu and gpu.

replication code:

>>> import torch
>>> device = torch.device("cuda:0")
>>> cpu_gen = torch.Generator(); cpu_gen.manual_seed(42)
>>> torch.randn(2, 2, generator=cpu_gen)
tensor([[0.3367, 0.1288],
        [0.2345, 0.2303]])
>>> gpu_gen = torch.Generator(device=device); gpu_gen.manual_seed(42)
>>> torch.randn(2, 2, generator=gpu_gen, device=device)
tensor([[ 0.1940,  2.1614],
        [-0.1721,  0.8491]], device='cuda:0')

so our random transform in clr will have different values on cpu and gpu.

[mattf1n]

Good heavens. We can either not worry about determinism, or have it to always generate on CPU (since this only happens once).

[themurtazanazir]

Yes we can generate the transform on the cpu, move it to gpu afterwards. However, our current experiments have initiated it on the gpu, so we can ignore it for now and fix it lateron

[mattf1n]

Shall we close this? Not essential to do it seem.

[themurtazanazir]

yes... seems non essential at this point. can reopen when needed.