Closed thangld201 closed 7 months ago
The reference model doesn't affect the proxy model's objective (it's a constant), so it's a question of how this affects the domain weights \alpha. In our implementation, we update \alpha as \alpha_new \propto \alpha exp(-eta max(excess loss, 0}), so when excess loss is 0, we'll just stop updating \alpha (and continue training the proxy model with this \alpha). We output the time-averaged \alpha, so this will slowly move the average towards this final value. Intuitively, this value of \alpha allowed us to quickly reduce the excess loss, so we should just continue with this \alpha for the rest of the compute budget remaining.
Thank for the wonderful work ! I have a question about Equation 1 in the paper. If the proxy model's parameters become (or get initialized) the same as the reference model, then training would converge and the learned domain weights might not be meaningful (since they do not matter now as loss = 0). Can you clarify @sangmichaelxie ?