Recommendations for computations stability

[arogozhnikov]

I've performed some tests for reversible modules, I think the following recommendations should be helpful:

• torch.inverse is not sufficiently accurate when using float, using if with double argument is preferred torch.inverse(x.double()).float(), this significantly reduces reconstruction error
• there is torch.slogdet method, which more accurate then manual log+abs+det: dlogdet = torch.log(torch.abs(torch.det(self.weight))) * pixels

Thanks for porting code to torch :)

[chaiyujin]

Thanks so much for you recommendations. I'll follow them and fix the code.

[wmjung]

The slogdet function returns a tuple of sign and absolute value of determinant. https://pytorch.org/docs/stable/torch.html#torch.slogdet I think you need to modify to get the value like below

torch.slogdet(self.weight)[1] * pixels

[chaiyujin]

oooh, my fault. It should be torch.logdet().

[arogozhnikov]

oooh, my fault. It should be torch.logdet().

@wmjung is right - you just need to take second value in output, because in paper you compute log-abs-det, that's exactly the second value of slogdet!

[chaiyujin]

Yes, you're right. I tried and fixed it.