Currently, the Tanh() bijection returns -inf as the log Jacobian determinant whenever any of the elements of the input tensor are roughly 6.0 or greater. This is because torch.tanh() returns exactly 1.0 at that point, causing the current implementation to pass 1 - 1 ** 2 = 0 to torch.log(), which of course returns -inf. This PR fixes the problem by adding a very small constant (1e-45) to the input to torch.log().
didriknielsen
commented
3 years ago
Currently, the Tanh() bijection returns -inf as the log Jacobian determinant whenever any of the elements of the input tensor are roughly 6.0 or greater. This is because torch.tanh() returns exactly 1.0 at that point, causing the current implementation to pass 1 - 1 ** 2 = 0 to torch.log(), which of course returns -inf. This PR fixes the problem by adding a very small constant (1e-45) to the input to torch.log().