search

yaringal / multi-task-learning-example

A multi-task learning example for the paper https://arxiv.org/abs/1705.07115
MIT License
838 stars 205 forks source link

This is a lucky demo When I change the data generation process, the prediction of the variance is wrong #12

Open conan-ux opened 3 years ago

conan-ux commented 3 years ago

I think this is a lucky demo. When I chenge the data generation code, the optimization will be guided wrongly and the variance prediction is wrong.

So I think this uncertainty method only works at a situation that: the value of the diff is close to precsion and log variance. If their value are not at the same scale, the method will broke.

`

def gen_data(N):

X = np.random.randn(N, Q)

w1 = 2.*1e2

b1 = 8.*1e2

sigma1 = 10  # ground truth

Y1 = X.dot(w1) + b1 + sigma1 * np.random.randn(N, D1)

w2 = 3*1e2

b2 = 3*1e2

sigma2 = 1*1e2 # ground truth

Y2 = X.dot(w2) + b2 + sigma2 * np.random.randn(N, D2)

return X, Y1, Y2

`

ichxw commented 3 years ago

@conan-ux I got a similar issue. Have you tried tuning the learning rate?

RuixuanDai commented 3 years ago

Try to increase epochs or tune the learning rate.