Closed michalwols closed 4 years ago
Without it large logits lead to infs and nans due to torch.log(0)
torch.log(0)
before:
asl = AsymmetricLoss() asl(torch.Tensor([[-100]]), torch.Tensor([[0]])) >> tensor(nan)
after:
asl = AsymmetricLoss(eps=0) asl(torch.Tensor([[-100]]), torch.Tensor([[0]])) >> tensor(nan) asl = AsymmetricLoss() asl(torch.Tensor([[-100]]), torch.Tensor([[0]])) >> tensor(-0.)
Thanks @michalwols we didnt encounter these instabilities in our training, but indeed it is important protection.
Without it large logits lead to infs and nans due to
torch.log(0)
before:
after: