search

imtiazziko / LaplacianShot

Laplacian Regularized Few Shot Learning
79 stars 8 forks source link

can we remove ylog(y)? #2

Closed amsword closed 3 years ago

amsword commented 3 years ago

in Eqn. 4, it seems like we can remove y log(y). Based on Eqn. 6, Eqn. 4 will be upper bounded by a linear function of Y. Since each element of Y has to be 0 or 1, the optimal solution of teh upper bound could be easily solved by assigning the y whose linear coefficient is the smallest as 1, and all others as 0.

imtiazziko commented 3 years ago

Hi, Thanks for this good and interesting question.

Yeah, it could be done as well. However, in our case this negative log term is helping by implicitly handling the non-negativity constraints for simplex and duel variable for simplex constraints, which turns into a nice softmax update for each assignment. Also this term convexify the objectives which facilitates better optimization with better objective value.