Open zengzix2 opened 2 weeks ago
No there is no optimal policy for setting it for the purpose of this assignment. Better ways of finding a good step size are mentioned in the README (Newton method etc) that address this issue. You might be setting the step size too big for the gradient descent. Try lowering it, if it doesn't help then it might be a bug in your line search or projection.
I'm defaulting initial step size to 1000, it's a slow movement toward the query point. Is this the expected behavior?
yes it would be slow, but there shouldn't be too much oscillation and it should not diverge.
When I try to drag the lightbulb around, the whole body of the lamp moves back and forth around the query point with a large amplitude, unlike the smooth transitions in the handout .gif. (It does slow down and converge, though.)
I understand that it is trying to find minimum loss with gradient descent, but is this the expected behavior? Also, is there an optimal setting for initializing max_step? Or should we just hard-code it to a number?