oxcsaml2019 / multitask-learning

16 stars 3 forks source link

Why multiplying the uncertainty by 0.5? #2

Open OmriKaduri opened 3 years ago

OmriKaduri commented 3 years ago

Hi, at your loss function you calculate:

loss += torch.exp(-sem_uncertainty) * sem_loss + 0.5 * sem_uncertainty

Where sem_uncertainty should be the log variance of the semantic segmentation task, but why do you multiply it by 0.5?

As can be seen in the original paper, there is no divsion of that component: image

3xWangDot commented 3 years ago

@OmriKaduri here is my understanding: Assume the sigma in the original paper is x, and sigma in the code is y, then we will have log(x)=0.5y, then x=e^(0.5y). After that, taking x=e^(0.5y) into the original formulation, x^(-2) will become e^(-y), so the original formulation: x^(-2)loss+log(x) will become: e^(-y)loss+0.5y