search

composable-models / llm_multiagent_debate

ICML 2024: Improving Factuality and Reasoning in Language Models through Multiagent Debate
304 stars 42 forks source link

Discussion about adding a Multi-Agent (Reflection) experiment #8

Closed Tebmer closed 11 months ago

Tebmer commented 11 months ago

Hi, your work is really perfect and elaborate, inspiring me a lot!

But I am worried about that the improvement of Multi-Agent (Debate) may not totally be attributed to the "Debate" of multi-agent, but the "reflection" of each agent plus majority.

So, is it necessary to to add a new method "Multi-Agent (Reflection)" (i.e. reflection for each agent and then majority) to eliminate the influence of "self-reflection"?

yilundu commented 11 months ago

Hi,

I think that would be an interesting baseline to potentially add in the future -- our work was developed concurrently with self-reflection and thus we only added comparison to the base version of the approach.