search

madaan / self-refine

LLMs can generate feedback on their work, use it to improve the output, and repeat this process iteratively.
https://selfrefine.info
Apache License 2.0
596 stars 48 forks source link

Code optimization #5

Closed juny116 closed 1 year ago

juny116 commented 1 year ago

Hello,I am really interested in your fine work and trying to reproduce the results!

Can you share the prompts and examples for the code optimization? I am having hard time reproducing the code optimization results. I have an additional question regarding code optimization in your paper. Specifically, I'm interested in how you calculated the percentage of programs that were optimized. When reproducing the results, I noticed that some codes performed worse, while others actually showed improvement.

madaan commented 1 year ago

Hi @juny116 , sorry about the late response.

The prompts and the code has been added. For evaluation, you'll need to follow evaluation setup here.