SELF-REFINE: Iterative Refinement with Self-Feedback

[Aidenzich]

https://arxiv.org/pdf/2303.17651.pdf

[Aidenzich]

Performance

• Code Optimization: 24.8 -> 27.5 (10.8%)
• Sentiment Reversal: 0 -> 43.2
• Acronym Generation: 48 -> 56.4 (17.5%)

Generic feedback: like improving the efficiency of the code, lacks this precision and direction.

[Aidenzich]

Influence of the Iteration

[Aidenzich]

Appendix. GPT-4 Evaluation

These are essentially what you'll see in the gpt-prompt-engineer repo and the automatic evaluation metrics at my company. It appears that pair-to-pair comparison is more useful than simply using prompts for evaluation or a fine-tuned classifier.