Open Josh00-Lu opened 6 months ago
Even after ~ 1 billion self-play, over 1000 checkpoints, the model seems still not converge
Thanks for your wonderful project. May I ask how do you judge the convergency of a self-play game model? Are there any evaluation metrics recommended?
I don't think evaluating with rewards (or utilities) is a good choice, because it is continuously self-improving on both sides at the training stage.
No good way from my pov, you can try train one and see it's performance vs slumbot, but this simulator don't matches the rule of ACPC. You need to modify the simulator to do that.
Thanks for your wonderful project. May I ask how do you judge the convergency of a self-play game model? Are there any evaluation metrics recommended?
I don't think evaluating with rewards (or utilities) is a good choice, because it is continuously self-improving on both sides at the training stage.