Hi,
Thanks for this great work. In Eq. 12/13 you have a log(variance) that might go negative for variance<1. Assuming, variance to be always +ve can we modify the Eq. 12/13 to log(1+variance) to prevent negative loss and allowing better optimization?
Will this modification hamper training or net output during inference?
Hi,
I never tried, because the negative loss never gave issues in my experiments.
You can use sigmoids over the outputs, but I think your solution should work as well.
Hi, Thanks for this great work. In Eq. 12/13 you have a
log(variance)
that might go negative for variance<1. Assuming, variance to be always +ve can we modify the Eq. 12/13 tolog(1+variance)
to prevent negative loss and allowing better optimization? Will this modification hamper training or net output during inference?