Reflexion: Language Agents with Verbal Reinforcement Learning

[Aidenzich]

https://arxiv.org/pdf/2303.11366.pdf

Verbal reinforcement learning : a method where AI agents learn and improve their actions based on descriptive, language-based feedback, enhancing their decision-making through reflection and adaptation.

It Builds upon ReAct, presenting an incremental advancement by integrating verbal or linguistic feedback (evaluator) into the reinforcement learning process.

[Aidenzich]

Perfomance

[Aidenzich]

The workflow of ReAct: Reasoning -> Act -> Observe The workflow of this paper: Reasoning -> Act -> Evaluate -> Reflect -> Adapt

Act: The agent performs an action in the environment based on its current policy and knowledge. Evaluate: The action's outcome is assessed, often by an evaluator model, to determine its success or failure. Reflect: The agent receives verbal feedback about the action, which includes reflections on why the action succeeded or failed and suggestions for improvement. Adapt: The agent incorporates this reflective feedback into its episodic memory, using it to inform and adjust future actions.

[Aidenzich]

Implementation