When I train the 2 stages, I find that the loss changes in a small oscillation from beginning to end even if it falls in a general. I think it is unrelated to the fix of learning rate for I just use the origin learing rate in the code and the loss keep falling in the overall trend. Why is the loss always jumping?
When I train the 2 stages, I find that the loss changes in a small oscillation from beginning to end even if it falls in a general. I think it is unrelated to the fix of learning rate for I just use the origin learing rate in the code and the loss keep falling in the overall trend. Why is the loss always jumping?