Closed mingliangzhang2018 closed 2 years ago
I agree that tuning weight decay improves performance significantly! We focused on weight decay since it is a weight balancing method, which is in the direction of our work.
I would like to learn what additional hyperparameters you found impactful in the LTR problem.
This research field is very messy, and most people are playing around without getting to the bottom of the problem. The performance could be highly improved by just adjusting the hyperparameters of general networks, including the weight decay and others that you have not mentioned but used. I think the future direction is to solve the optimization problem of long-tailed learning.