Sampling of sigma

[pixeli99]

Hi, it seems that a simple log-normal distribution should be used to sample sigma. https://github.com/CiaraStrawberry/svd-temporal-controlnet/blob/bdd2e8fd833e2bbd83f235b6009154e4126924ad/train_svd.py#L179

Replace it with:

def rand_log_normal(shape, loc=0., scale=1., device='cpu', dtype=torch.float32):
    """Draws samples from an lognormal distribution."""
    u = torch.rand(shape, dtype=dtype, device=device) * (1 - 2e-7) + 1e-7
    return torch.distributions.Normal(loc, scale).icdf(u).exp()

Relevant discussions can be found here. https://github.com/pixeli99/SVD_Xtend/issues/21

[jiangzhengkai]

@pixeli99 Can you provide some references for rand_log_normal function? Any paper?