This paper uncovers that the judgement consistency of LLM dramatically decreases when confronted with disruptions like questioning, negation, or misleading, even though its previous judgments were correct. It also explores several prompting methods to mitigate this issue and demonstrates their effectiveness.
Hi,
I have read your insightful paper and found it to be a valuable contribution to the field.
I would like to kindly suggest adding our recent work to your survey.
📄 Paper: Ask Again, Then Fail: Large Language Models' Vacillations in Judgement
This paper uncovers that the judgement consistency of LLM dramatically decreases when confronted with disruptions like questioning, negation, or misleading, even though its previous judgments were correct. It also explores several prompting methods to mitigate this issue and demonstrates their effectiveness.
Thank you for your consideration! :)