trotsky1997
commented
3 days ago Early stopping based on the check function is a technique used for validation. However, MCTSR can still operate without a check function or ground truth for early stopping. For black-box optimization tasks, you can replace the check function with your black-box objective and early-stopping conditions. For open-domain tasks, it is necessary to redesign the early stopping or optimal node selection strategy, such as using max-Q or length-normalized logit ranking. However, these methods currently face the risk of performance degradation to suboptimal solutions. If you have designed a new and suitable stopping function or optimal node selection strategy for your task, please let me know. I would be delighted to incorporate it.
Hi, the paper was a very interesting read and this technique seems to have a lot of potential. However, looking at the code - if I understand it correctly - I have noticed that a significant portion of it is dependent on the knowledge of the correct answer - the 'ground truth'. If this knowledge is not available to the program until the final result validation, how does the program perform then? Thanks.