search

yaringal / multi-task-learning-example

A multi-task learning example for the paper https://arxiv.org/abs/1705.07115
MIT License
838 stars 205 forks source link

Final Loss equation #1

Closed hardianlawi closed 6 years ago

hardianlawi commented 6 years ago

Hi @yaringal !

I have read the paper and it is really amazing. Thanks for your team's hard-work!

However, I have a question regarding the final equation and also your keras implementation.

image

The equation above has 1/2 multiplied by the loss, but you didn't include it in the keras implementation.

I tried experimenting on it, and included 1/2 in the loss function, but it couldn't converge. I am wondering if the problem is in the paper or the keras implementation, because if I exclude the 1/2, it converges to the ground-truth std.

Best regards,

Hardian

yaringal commented 6 years ago

Please have a look at the updated NIPS version of the paper which has some typos corrected. Yarin

hardianlawi commented 6 years ago

hi @yaringal thanks for your reply

Are you referring to this? I can't seem to find the one with the same title

yaringal commented 6 years ago

Ah apologies this is not updated online yet. It will be soon. The equation above has a type - the implementation is correct. Yarin

divyansha commented 5 years ago

According to the most recent version, the losses should still be multiplied by 1/2:

screen shot 2019-01-14 at 6 00 26 pm

Why is a 1/2 missing from your code?

divyansha commented 5 years ago

I did the math from scratch and it does seem like there is agreement between code and the most recent version of paper.

Apologies for the trouble.

John1231983 commented 5 years ago

Here you are https://arxiv.org/pdf/1703.04977.pdf

Screenshot from 2019-03-13 10-09-14

I am implementing it in pytorch. If you have done it, please share it

John1231983 commented 5 years ago

@hardianlawi : I reproduced the result from keras using pytorch. YOu can look at https://colab.research.google.com/drive/1_zsmQguerz0iy0J9Uu2Cs7oEHhj0QoXH

However, the sigma2 does not work. Do you have any suggestion @yaringal ?

Banyueqin commented 5 years ago

@John1231983 you need to fix the code webwxgetmsgimg

John1231983 commented 5 years ago

@Banyueqin : Thanks. But this is result after fix it


2627.982177734375
749.9542846679688
538.127685546875
401.61016845703125
308.06640625
241.04234313964844
191.45272827148438
153.8364715576172
124.77854919433594
102.02399444580078
84.0322265625
69.68601989746094
58.185302734375
48.94459533691406
41.494354248046875
35.47389221191406
30.60963249206543
26.680410385131836
23.508365631103516
20.950298309326172
tensor([8.7028]) tensor([5.3088]) 18.911481857299805
Banyueqin commented 5 years ago

@John1231983
image

result: image

John1231983 commented 5 years ago

No. We expect result is 10 and 0.

lilaczhou commented 5 years ago

@hardianlawi My reuslt goes to negative number after some steps,and I think it results from log_var(sigma),would you mind give me some suggestions? image