madaan
commented
1 year ago Sorry about the late response.
No matter what the output from the LM is, the LM is prompted again with the same question and the generated code/text (code+comments), until the LM itself says "it is correct", with a maximum of max_attempts for each question?
Right.
so if the model outputs "it is correct"
In such cases, the self-refine loop can stop, see line 3 of the algorithm.
Hello! First of all this is a super nice paper.
I am trying to wrap my head around the concept of the paper. What I don't understand is this: No matter what the output from the LM is, the LM is prompted again with the same question and the generated code/text (code+comments), until the LM itself says "it is correct", with a maximum of max_attempts for each question? The paper reports improvements over 5 iterations, so if the model outputs "it is correct" the same output is used for the next iteration? Just want to make sure I understood this correctly.