Awesome LLM-Powered Agent

Thanks to the impressive planning, reasoning, and tool-calling capabilities of Large Language Models (LLMs), people are actively studying and developing LLM-powered agents. These agents are possible to autonomously (and collaboratively) solve complex tasks, or simulate human interactions.

Our goal with this project is to build an exhaustive collection of awesome resources relevant to LLM-powered agents encompassing papers, repositories, and more. We strive to keep these updated regularly and continuously. We greatly appreciate any contributions via PRs, issues, emails, or other methods.

Note that this repository is not under active maintenance. It mainly contains papers that appear before Oct. 2023, with several further papers. If you would like to have your paper included, please feel free to initiate a pull request.

• Papers
  • Autonomous Task Solver
  • General Reasoning & Planning & Tool Using
  • Multi-Agent Cooperation
  • Framework & Open-Source
  • Application
    • Web Agents
    • RL Agents
    • Robotics & Embodied AI
    • Gaming & Role-Playing
    • Other Applications
  • Trustworthy
  • Human Interaction Simulation
  • Human-Agent Interaction
  • Agents-Powered LLMs
  • Benchmark
  • Survey & Tutorial
• Open-Source Projects
  • Autonomous Task Solver Projects
  • Multi-Agent Simulation Projects
• Perspectives
• Other Related Sources
• Acknowledgement

Papers

🔥 for papers with >50 citations or repositories with >200 stars.\ 📖 for papers accepted by reputed conferences/journals.

Autonomous Task Solver

General Reasoning & Planning & Tool Using

• 🔥 [May 2024] "Offline Training of Language Model Agents with Functions as Learnable Weights ." Shaokun Zhang (PSU) et al. ICML 2024. [paper] [code]
• 🔥 [Mar 2024] "Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models ." Zehui Chen (USTC) et al. arXiv. [paper] [code] [project page]
• [Dec 2023] "CLOVA: A Closed-LOop Visual Assistant with Tool Usage and Update." Zhi Gao (BIGAI) et al. arXiv. [paper] [code] [project page]
• 📖 [Dec 2023] "SayCanPay: Heuristic Planning with Large Language Models using Learnable Domain Knowledge." Rishi Hazra et al. AAAI 2024 [paper] [code] [project page]
• [Oct 2023] "Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models." Andy Zhou (UIUC) et al. arXiv. [paper] [code] [project page]
• 🔥📖 [Oct 2023] "Large Language Models Cannot Self-Correct Reasoning Yet." Jie Huang (UIUC) et al. ICLR 2024. [paper]
• [Oct 2023] "Adapting LLM Agents Through Communication." Kuan Wang (GaTech & Microsoft) et al. arXiv. [paper]
• 📖 [Oct 2023] "ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search" Yuchen Zhuang (GaTech & Adobe) et al. ICLR 2024. [paper]
• 📖 [Sep 2023] "AVIS: Autonomous Visual Information Seeking with Large Language Models." Ziniu Hu (Google) et al. NeurIPS 2023. [paper]
• [Sep 2023] "Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency." Zhihan Liu (Northwestern) et al. arXiv. [paper] [code] [project page]
• [Sep 2023] "Self-driven Grounding: Large Language Model Agents with Automatical Language-aligned Skill Learning." Shaohui Peng (CAS) et al. arXiv. [paper]
• [Aug 2023] "ExpeL: LLM Agents Are Experiential Learners." Andrew Zhao (THU) et al. arXiv. [paper]
• [Aug 2023] "Synergistic Integration of Large Language Models and Cognitive Architectures for Robust AI: An Exploratory Analysis." Oscar J. Romero (CMU) et al. arXiv. [paper]
• [Aug 2023] "Dynamic Planning with a LLM." Gautier Dagan (U of Edinburgh) et al. arXiv. [paper] [code]
• [Aug 2023] "Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization." Weiran Yao (Salesforce) et al. arXiv. [paper]
• 🔥 [May 2023] "ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models." Binfeng Xu et al. arXiv. [paper] [code]
• 📖 [May 2023] "SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks." Bill Yuchen Lin (AI2) et al. NeurIPS 2023. [paper] [code] [project page]
• 📖 [May 2023] "AdaPlanner: Adaptive Planning from Feedback with Language Models." Haotian Sun (GaTech) et al. NeurIPS 2023. [paper] [code]
• 🔥📖 [May 2022] "Reasoning with Language Model is Planning with World Model." Shibo Hao (UCSD) et al. EMNLP 2023. [paper] [code] [project page]
• 📖 [May 2023] "Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning." Lin Guan (ASU) et al. NeurIPS 2023. [paper] [code] [project page]
• 📖 [May 2023] "ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models." Zhipeng Chen (RUC) et al. EMNLP 2023 Findings. [paper] [code]
• [May 2023] "CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing." Zhibin Gou (THU & Microsoft) et al. arXiv. [paper] [code]
• 🔥 [Apr 2023] "LLM+P: Empowering Large Language Models with Optimal Planning Proficiency." Bo Liu (UT Austin) et al. arXiv. [paper] [code]
• 🔥📖 [Mar 2023] "Reflexion: Language Agents with Verbal Reinforcement Learning." Noah Shinn (Northeastern) et al. NeurIPS 2023. [paper] [code]
• 📖 [Dec 2022] "Don’t Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments" Yu Gu (OSU) et al. ACL 2023. [paper] [code]
• 🔥📖 [Oct 2022] "ReAct: Synergizing Reasoning and Acting in Language Models." Shunyu Yao (Princeton & Google Brain) et al. ICLR 2023. [paper] [code] [project page]

Multi-Agent Cooperation

• [Aug 2024] "MegaAgent: A Practical Framework for Autonomous Cooperation in Large-Scale LLM Agent Systems" Qian Wang (NUS) et al.* arXiv. [paper]
• [May 2024] "Conformity, Confabulation, and Impersonation: Persona Inconstancy in Multi-Agent LLM Collaboration." Razan Baltaji (UIUC) et al.* arXiv. [paper] [code]
• [April 2024] "CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solving." Pei Chen (TAMU) et al.* NAACL 2024. [paper] [code]
• 📖 [Jan 2024] "L2MAC: Large Language Model Automatic Computer for Extensive Code Generation." Samuel Holt (Cambridge) et al. ICLR 2024. [paper] [code] [project page]
• [Oct 2023] "Evaluating Multi-Agent Coordination Abilities in Large Language Models." Saaket Agashe (UCSC) et al. arXiv. [paper]
• [Oct 2023] "Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with Agent Team Optimization." Zijun Liu (THU & Stanford) et al. arXiv. [paper] [code]
• [Oct 2023] "Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View." Jintian Zhang (ZJU) et al. arXiv. [paper] [code]
• [Oct 2023] "Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration." Qiushi Sun (Shanghai AI Lab & NUS) et al. arXiv. [paper] [code]
• [Sep 2023] "LLM-Deliberation: Evaluating LLMs with Interactive Multi-Agent Negotiation Games." Sahar Abdelnabi (CISPA) et al. arXiv. [paper] [code]
• [Sep 2023] "Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?" Yongchao Chen (MIT & Harvard) et al. arXiv. [paper] [code] [project page]
• [Sep 2023] "ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs." Justin Chih-Yao Chen (UNC Chapel Hill) et al. arXiv. [paper] [code]
• [Sep 2023] "MindAgent: Emergent Gaming Interaction." Xiaojian Ma (BIGAI) et al. arXiv. [paper] [code] [project page]
• [Aug 2023] "ProAgent: Building Proactive Cooperative AI with Large Language Models." Ceyao Zhang (CUHK & PKU) et al. arXiv. [paper] [project page]
• 🔥 [Aug 2023] "AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents." Weize Chen (THU) et al. arXiv. [paper] [code]
• [Aug 2023] "GPT-in-the-Loop: Adaptive Decision-Making for Multiagent Systems." Nathalia Nascimento (U of Waterloo) et al. arXiv. [paper]
• [Aug 2023] "How susceptible are LLMs to Logical Fallacies?" Amirreza Payandeh (GMU & Vail Systems) et al. arXiv. [paper] [code]
• [Aug 2023] "ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate." Chi-Min Chan (THU) et al. arXiv. [paper] [code]
• [Aug 2023] "LLM As DBA." Xuanhe Zhou (THU) et al. arXiv. [paper] [code]
• [Aug 2023] "Gentopia: A Collaborative Platform for Tool-Augmented LLMs." Binfeng Xu et al. arXiv. [paper] [code] [project page]
• 🔥 [Aug 2023] "MetaGPT: Meta Programming for Multi-Agent Collaborative Framework." Sirui Hong (DeepWisdom) et al. arXiv. [paper] [code]
• [Jul 2023] "PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations." Ruosen Li (UT Dallas) et al. arXiv. [paper] [project page][code]
• [Jul 2023] "Unleashing Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration." Zhenhailong Wang (UIUC & MSRA) et al. arXiv. [paper] [code]
• [Jul 2023] "RoCo: Dialectic Multi-Robot Collaboration with Large Language Models." Mandi Zhao (Columbia) et al. arXiv. [paper] [code] [project page]
• [Jul 2023] "Wireless Multi-Agent Generative AI: From Connected Intelligence to Collective Intelligence." Hang Zou (Technology Innovation Institute, UAE) et al. arXiv. [paper]
• [Jul 2023] "Building Cooperative Embodied Agents Modularly with Large Language Models." Hongxin Zhang (UMass) et al. arXiv. [paper] [code] [project page]
• [Jun 2023] "RestGPT: Connecting Large Language Models with Real-World Applications via RESTful APIs." Yifan Song (PKU) et al. arXiv. [paper] [project page]
• [Jun 2023] "Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents." Yashar Talebirad (UAlberta) et al. arXiv. [paper]
• [May 2023] "Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate." Tian Liang (THU & Tencent) et al. arXiv. [paper] [code]
• 🔥 [May 2023] "Large Language Models as Tool Makers." Tianle Cai (Deepmind & Princeton) et al. arXiv. [paper] [code]
• [May 2023] "Improving Factuality and Reasoning in Language Models through Multiagent Debate." Yilun Du (MIT) et al. arXiv. [paper] [code] [project page]
• [May 2023] "Agreement and Statistical Efficiency in Bayesian Perception Models." Yash Deshpande (MIT) et al. arXiv. [paper]
• [May 2023] "Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback." Yao Fu (U of Edinburgh) et al. arXiv. [paper] [code]

Framework & Open-Source

• 🔥 [Oct 2023] "OpenAgents: An Open Platform for Language Agents in the Wild." Tianbao Xie (HKU & XLang Lab) et al. arxiv. [paper] [code]
• 🔥 [Sep 2023] "AutoAgents: A Framework for Automatic Agent Generation." Guangyao Chen (PKU) et al. arXiv. [paper] [code]
• 🔥 [Sep 2023] "Agents: An Open-source Framework for Autonomous Language Agents." Wangchunshu Zhou (AI Waves) et al. arXiv. [paper] [code] [project page]
• 🔥 [Sep 2023] "Cognitive Architectures for Language Agents." Theodore Sumers (Princeton) et al. arXiv. [paper] [repo]
• 🔥 [Aug 2023] "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework." Qingyun Wu et al. arXiv. [paper] [code] [project page]
• 🔥 [Aug 2023] "FastAgency: The fastest way to bring multi-agent workflows to production." Davor Runje et al. [code] [project page]

Application

Web Agents

• [Sep 2023] "You Only Look at Screens: Multimodal Chain-of-Action Agents." Zhuosheng Zhang (SJTU) et al. arXiv. [paper] [code]
• [Sep 2023] "LASER: LLM Agent with State-Space Exploration for Web Navigation." Kaixin Ma (Tencent) et al. arXiv. [paper] [code]
• 🔥 [Jul 2023] "WebArena: A Realistic Web Environment for Building Autonomous Agents." Shuyan Zhou (CMU) et al. arXiv. [paper] [code] [project page]
• [Jul 2023] "A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis." Izzeddin Gur (DeepMind) et al. arXiv. [paper]
• 🔥📖 [Jun 2023] "Mind2Web: Towards a Generalist Agent for the Web." Xiang Deng (OSU) et al. NeurIPS 2023. [paper] [code] [project page]
• [May 2023] "Augmenting Autotelic Agents with Large Language Models." Cédric Colas (MIT & Inria) et al. arXiv. [paper]
• [May 2023] "Mobile-Env: An Evaluation Platform and Benchmark for Interactive Agents in LLM Era." Danyang Zhang (SJTU) et al. arXiv. [paper] [code]
• 📖 [Apr 2023] "Emergent autonomous scientific research capabilities of large language models." Daniil A. Boiko (CMU) et al. arXiv. [paper]
• [Mar 2023] "Language Models can Solve Computer Tasks." Geunwoo Kim (UCI) et al. arXiv. [paper] [code] [project page]
• 📖 [Jul 2022] "WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents." Shunyu Yao (Princeton) et al. NeurIPS 2022. [paper] [code] [project page]

RL Agents

• [May 2024] "Agentic Skill Discovery." Xufeng Zhao (UHH) et al. arXiv. [paper] [code] [project page]
• [May 2024] "Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models." Cong Lu (UBC) et al. arXiv. [paper]
• [Nov 2023] "Lafite-RL: "Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Models" Kun Chu (UHH) et al. CRL WS @ CoRL 2023. [Paper]
• [Oct 2023] "Motif: Intrinsic Motivation from Artificial Intelligence Feedback." Martin Klissarov (Mila & Meta & McGill) et al. arXiv. [paper]
• [Sep 2023] "RLAdapter: Bridging Large Language Models to Reinforcement Learning in Open Worlds." Wanpeng Zhang (PKU) et al. arXiv. [paper]
• [Aug 2023] "LaGR-SEQ: Language-Guided Reinforcement Learning with Sample-Efficient Querying." Thommen George Karimpanal (Deakin University) et al. arXiv. [paper] [code]
• [Jul 2023] "Dialogue Shaping: Empowering Agents through NPC Interaction." Wei Zhou (GaTech) et al. arXiv. [paper]
• [Jul 2023] "Towards A Unified Agent with Foundation Models." Norman Di Palo (ICL & DeepMind) et al. Reincarnating RL @ ICLR 2023. [paper]
• 📖 [Jun 2023] "Large Language Model Is Semi-Parametric Reinforcement Learning Agent." Danyang Zhang (SJTU) et al. NeurIPS 2023. [paper]
• [May 2023] "Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning." Wenhao Li (CUHK) et al. arXiv. [paper]

Robotics & Embodied AI

• [Nov 2024] "LABOR Agent: Large Language Models for Orchestrating Bimanual Robots", Kun Chu (UHH) et al. Humanoids 2024. [Paper] [Website], [Code]
• [Jul 2024] "Odyssey: Empowering Agents with Open-World Skills." Shunyu Liu (Zhejiang University) et al. arXiv. [paper] [code]
• [Nov 2023] "LEO: An Embodied Generalist Agent in 3D World." Xiaojian Ma (BIGAI) et al. arXiv. [paper] [code] [project page]
• [Nov 2023] "JARVIS-1: Open-world Multi-task Agents with Memory-Augmented Multimodal Language Models." Zihao Wang (PKU) et al. arXiv. [paper] [code] [project page]
• [Oct 2023] "Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond." Liang Chen (PKU) et al. arXiv. [paper] [code] [project page]
• [Oct 2023] "LANCAR: Leveraging Language for Context-Aware Robot Locomotion in Unstructured Environments." Chak Lam Shek (UMD) et al. arXiv. [paper] [project page]
• [Sep 2023] "LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent." Jianing Yang (UMich) et al. arXiv. [paper] [code] [project page]
• [Sep 2023] "SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models." Shyam Sundar Kannan (Purdue) et al. arXiv. [paper] [project page]
• [Sep 2023] "Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents." Ziyi Yang et al. arXiv. [paper] [code & video]
• [Sep 2023] "SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environments." Abhinav Rajvanshi (SRI International) et al. arXiv. [paper]
• [Sep 2023] "Developmental Scaffolding with Large Language Models." M. Batuhan Celik (Bogazici University) et al. arXiv. [paper]
• [Jul 2023] "March in Chat: Interactive Prompting for Remote Embodied Referring Expression." Yanyuan Qiao (Adelaide University) et al. arXiv. [paper] [code]
• [Aug 2023] "A^2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models." Peihao Chen (SCUT) et al. arXiv. [paper]
• [Jul 2023] "Embodied Task Planning with Large Language Models." Zhenyu Wu (BUPT) et al. arXiv. [paper] [code] [project page]
• [Jun 2023] "Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach." Bin Hu (Zhejiang Lab) et al. arXiv. [paper] [code]
• 🔥 [May 2023] "Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory." Xizhou Zhu (THU & SenseTim) et al. arXiv. [paper] [code]
• 🔥 [May 2023] "Voyager: An Open-Ended Embodied Agent with Large Language Models." Guanzhi Wang (NVIDIA & Caltech) et al. arXiv. [paper] [code] [project page]
• [May 2023] "Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents." Yue Wu (CMU) et al. arXiv. [paper]
• 📖 [Feb 2023] "Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents." Zihao Wang (PKU) et al. NeurIPS 2023. [paper] [code]
• [Feb 2023] "Collaborating with language models for embodied reasoning." Ishita Dasgupta (DeepMind) et al. LaReL @ NeurIPS 2022. [paper]
• [Jan 2023] "Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling." Kolby Nottingham (UCI) et al. ICML 2023. [paper] [code] [project page]
• 📖 [Dec 2022] "LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models." Chan Hee Song (OSU) et al. ICCV 2023. [paper] [project page]

Gaming & Role-Playing

• 📖 [May 2024] "TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models." Jaewoo Ahn (SNU) et al. Findings of ACL 2024. [paper] [code] [project page]
• [Oct 2023] "From Text to Tactic: Evaluating LLMs Playing the Game of Avalon." Jonathan Light (RPI) et al. arXiv. [paper] [code]
• [Oct 2023] "Ruffle&Riley: Towards the Automated Induction of Conversational Tutoring Systems." Robin Schmucker (CMU) et al. arXiv. [paper]
• [Oct 2023] "Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation." Shenzhi Wang (THU) et al. arXiv. [paper]
• [Sep 2023] "MindAgent: Emergent Gaming Interaction." Xiaojian Ma (BIGAI) et al. arXiv. [paper] [code] [project page]
• [Sep 2023] "Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4." Jiaxian Guo (U of Tokyo) et al. arXiv. [paper] [code]
• [Aug 2023] "Ambient Adventures: Teaching ChatGPT on Developing Complex Stories." Zexin Chen (GaTech) et al. arXiv. [paper]
• [Jul 2023] "Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models." Yuanzhi Liang (UTS) et al. arXiv. [paper]
• [May 2023] "Role-Play with Large Language Models." Murray Shanahan (DeepMind & ICL) et al. arXiv. [paper]
• [May 2023] "clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents." Kranti Chalamalasetti (University of Potsdam) et al. arXiv. [paper] [code]
• [Apr 2023] "Towards autonomous system: flexible modular production system enhanced with large language model agents." Yuchen Xia (University of Stuttgart) et al. arXiv. [paper] [code]
• 🔥📖 [Mar 2023] "CAMEL: Communicative Agents for "Mind" Exploration of Large Scale Language Model Society." Guohao Li (KAUST) et al. NeurIPS 2023. [paper] [code] [project page]

Other Applications

• [May 2024] "AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments" Samuel Schmidgall (JHU & Stanford) et al. arXiv. [paper] [code] [project page]
• [Apr 2024] "ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoning" Ling Yue (RPI) et al. arXiv. [paper] [code]
• [Jan 2024] "EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Records." Wenqi Shi (GaTech) et al. arXiv. [paper] [code] [project page]
• [Oct 2023] "OptiMUS: Optimization Modeling Using mip Solvers and large language models." Ali AhmadiTeshnizi (Stanford) et al. arXiv. [paper] [code]
• [Oct 2023] "An evolutionary model of personality traits related to cooperative behavior using a large language model." Reiji Suzuki (Nagoya University) et al. arXiv. [paper]
• [Oct 2023] "Large Language Model (LLM) as a System of Multiple Expert Agents: An Approach to solve the Abstraction and Reasoning Corpus (ARC) Challenge." John Chong Min Tan (NUS) et al. arXiv. [paper]
• [Oct 2023] "A Language-Agent Approach to Formal Theorem-Proving." Amitayush Thakur (UT Austin) et al. arXiv. [paper]
• [Oct 2023] "Conversational Health Agents: A Personalized LLM-Powered Agent Framework." Mahyar Abbasian (UCI) et al. arXiv. [paper]
• [Oct 2023] "OceanGPT: A Large Language Model for Ocean Science Tasks." Zhen Bi (ZJU & Donghai Lab) et al. arXiv. [paper] [project page]
• [Oct 2023] "Voice2Action: Language Models as Agent for Efficient Real-Time Interaction in Virtual Reality." Yang Su (Cornell Tech). arXiv. [paper]
• 🔥 [Sep 2023] "ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving." Zhibin Gou (THU & Microsoft) et al. arXiv. [paper] [code] [project page]
• [Sep 2023] ""Teach AI How to Code": Using Large Language Models as Teachable Agents for Programming Education." Hyoungwook Jin (KAIST) et al. arXiv. [paper]
• [Sep 2023] "SurrealDriver: Designing Generative Driver Agent Simulation Framework in Urban Contexts based on Large Language Model." Ye Jin (THU) et al. arXiv. [paper]
• [Sep 2023] "Large Language Models as Agents in the Clinic." Nikita Mehandru (UC Berkeley) et al. arXiv. [paper]
• [Sep 2023] "An Appraisal-Based Chain-Of-Emotion Architecture for Affective Language Model Game Agents." Maximilian Croissant (UOY) et al. arXiv. [paper]
• [Sep 2023] "Unleashing the Power of Graph Learning through LLM-based Autonomous Agents." Lanning Wei (CAS & 4Paradigm) et al. arXiv. [paper]
• [Sep 2023] "TradingGPT: Multi-Agent System with Layered Memory and Distinct Characters for Enhanced Financial Trading Performance." Yang Li (SIT) et al. arXiv. [paper]
• 🔥 [Sep 2023] "ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models." Chenliang Li (Alibaba) et al. arXiv. [paper] [code] [demo]
• [Aug 2023] "Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations." Xu Huang (USTC) et al. arXiv. [paper] [code]
• [Aug 2023] "RecMind: Large Language Model Powered Agent For Recommendation." Yancheng Wang (ASU) et al. arXiv. [paper]
• [Aug 2023] "LLM Powered Sim-to-real Transfer for Traffic Signal Control." Longchao Da (ASU) et al. arXiv. [paper]
• [Aug 2023] "Out of the Cage: How Stochastic Parrots Win in Cyber Security Environments." Maria Rigaki (ČVUT) et al. arXiv. [paper] [code]
• [Aug 2023] "Is There Any Social Principle for LLM-Based Agents?" Jitao Bai (TJU) et al. arXiv. [paper]
• [Aug 2023] "ChatEDA: A Large Language Model Powered Autonomous Agent for EDA." Zhuolun He (CUHK & Shanghai AI Lab) et al. arXiv. [paper]
• [Aug 2023] "The Hitchhiker's Guide to Program Analysis: A Journey with Large Language Models." Haonan Li (UCR) et al. arXiv. [paper]
• [Jun 2023] "Towards Autonomous Testing Agents via Conversational Large Language Models." Robert Feldt (Chalmers University of Technology) et al. arXiv. [paper]
• [Apr 2023] "GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information." Qiao Jin, Yifan Yang, Qingyu Chen, Zhiyong Lu arXiv. [paper] [code]
• 🔥 [Mar 2023] "HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face." Yongliang Shen (ZJU & MSRA) et al. arXiv. [paper] [code]

Trustworthy

• [Feb 2024] "Can Large Language Model Agents Simulate Human Trust Behaviors?" Chengxing Xie (KAUST) et al. arXiv. [paper] [code] [project page]
• [Sep 2023] "Identifying the Risks of LM Agents with an LM-Emulated Sandbox" Yangjun Ruan (University of Toronto & Vector Institute) et al. arXiv. [paper] [code] [demo] [project page]
• [Aug 2023] "Enhancing Trust in LLM-Based AI Automation Agents: New Considerations and Future Challenges." Sivan Schwartz (IBM Research) et al. AutoMate @ IJCAI 2023. [paper]

Human Interaction Simulation

• [Mar 2024] "Emergence of Social Norms in Large Language Model-based Agent Societies." Siyue Ren (NWPU) et al. arXiv. [paper] [code]
• [Jan 2024] "Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models." Lucio La Cava (University of Calabria) et al. arXiv. [paper]
• 🔥 [Oct 2023] "SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents" Xuhui Zhou (CMU) et al.ICLR [paper]
• [Oct 2023] "CompeteAI: Understanding the Competition Behaviors in Large Language Model-based Agents." Qinlin Zhao (USTC) et al. arXiv. [paper]
• [Oct 2023] "Simulating Social Media Using Large Language Models to Evaluate Alternative News Feed Algorithms." Petter Törnberg (U of Amsterdam) et al. arXiv. [paper]
• [Oct 2023] "Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arena." Jiangjie Chen (FDU & AI2) et al. arXiv. [paper] [code] [project page]
• [Oct 2023] "Lyfe Agents: Generative agents for low-cost real-time social interactions." Zhao Kaiya (MIT) et al. arXiv. [paper]
• [Sep 2023] "Identifying the Risks of LM Agents with an LM-Emulated Sandbox" Yangjun Ruan (University of Toronto & Vector Institute) et al. arXiv. [paper] [code] [demo] [project page]
• [Sep 2023] "Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligence." Navid Ghaffarzadegan (Virginia Tech) et al. arXiv. [paper]
• [Aug 2023] "CGMI: Configurable General Multi-Agent Interaction Framework." Jinxin Shi (ECNU) et al. arXiv. [paper]
• [Aug 2023] "Exploring the Intersection of Large Language Models and Agent-Based Modeling via Prompt Engineering." Edward Junprung (UC Berkeley) et al. arXiv. [paper] [code]
• 🔥 [Aug 2023] "AgentSims: An Open-Source Sandbox for Large Language Model Evaluation." Jiaju Lin (PTA Studio & PSU) et al. arXiv. [paper] [code] [project page]
• [Jul 2023] "S^3: Social-network Simulation System with Large Language Model-Empowered Agents." Chen Gao (THU) et al. arXiv. [paper]
• [Jul 2023] "Are you in a Masquerade? Exploring the Behavior and Impact of Large Language Model Driven Social Bots in Online Social Networks." Siyu Li (SCU) et al. arXiv. [paper] [dataset]
• [Jul 2023] "Communicative Agents for Software Development." Chen Qian (THU) et al. arXiv. [paper]
• [Jul 2023] "Epidemic Modeling with Generative Agents." Ross Williams (Virginia Tech) et al. arXiv. [paper] [code]
• [Jul 2023] "To Infinity and Beyond: SHOW-1 and Showrunner Agents in Multi-Agent Simulations." Philipp Maas (Fable Studio) et al. preprint. [paper] [project page]
• [Jun 2023] "RecAgent: A Novel Simulation Paradigm for Recommender Systems." Lei Wang (RUC) et al. arXiv. [paper] [code]
• [May 2023] "Playing repeated games with Large Language Models." Elif Akata (U of Tübingen) et al. arXiv. [paper]
• [May 2023] "The Role of Summarization in Generative Agents: A Preliminary Perspective." Xiachong Feng (HIT) et al. arXiv. [paper]
• [Apr 2023] "Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models." Jimmy Wei (Cornell & Meta) et al. arXiv. [paper] [dataset] [code]
• 🔥 [Apr 2023] "Generative Agents: Interactive Simulacra of Human Behavior." Joon Sung Park (Stanford) et al. arXiv. [paper] [code]

Human-Agent Interaction

• [Oct 2023] "How AI Processing Delays Foster Creativity: Exploring Research Question Co-Creation with an LLM-based Agent." Yiren Liu (UIUC) et al. arXiv. [paper]
• [Aug 2023] "Quantifying the Impact of Large Language Models on Collective Opinion Dynamics." Chao Li (ZJU) et al. arXiv. [paper]
• [Aug 2023] "SAPIEN: Affective Virtual Agents Powered by Large Language Models." Masum Hasan (U of Rochester) et al. arXiv. [paper]
• [Jul 2023] "Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support." Zilin Ma (Harvard) et al. arXiv. [paper]

Agents-Powered LLMs

• [Oct 2023] "Agent Instructs Large Language Models to be General Zero-Shot Reasoners." Nicholas Crispino (WashU) et al. arXiv. [paper] [code]
• [Oct 2023] "ß-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis." Zishun Yu (UIC & ByteDance) et al. arXiv. [paper]
• 🔥 [May 2023] "Training Socially Aligned Language Models in Simulated Human Society." Ruibo Liu (Dartmouth) et al. arXiv. [paper] [code]
• 📖 [May 2023] "Language Models Meet World Models: Embodied Experiences Enhance Language Models." Jiannan Xiang (UCSD) et al. NeurIPS 2023. [paper] [code]

Benchmark

• 📖 [Jul 2024] "AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents." Harsh Trivedi (Stony Brook University) et al. ACL 2024. [paper] [code] [project page]
• [Dec 2023] "T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step." Zehui Chen (USTC, Shanghai AI Lab) et al. arXiv. [paper] [code] [project page]
• [Nov 2023] "MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration." Lin Xu et al.(NUS, ByteDance, Stanford & UC Berkeley) arXiv. [paper] [Project Page]
• [Oct 2023] "Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architectures." Thorsten Händler (FERNFH) et al. arXiv. [paper]
• [Oct 2023] "Benchmarking Large Language Models As AI Research Agents." Qian Huang (Stanford) et al. arXiv. [paper] [code]
• [Oct 2023] "MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use." Yue Huang (Lehigh University) et al. arXiv. [paper] [dataset]
• [Oct 2023] "SmartPlay : A Benchmark for LLMs as Intelligent Agents." Yue Wu (CMU & Microsoft) et al. arXiv. [paper] [code]
• [Sep 2023] "Identifying the Risks of LM Agents with an LM-Emulated Sandbox" Yangjun Ruan (University of Toronto & Vector Institute) et al. arXiv. [paper] [code] [demo] [project page]
• [Aug 2023] "BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents." Zhiwei Liu (Salesforce) et al. arXiv. [paper] [code]
• 🔥 [Aug 2023] "AgentBench: Evaluating LLMs as Agents." Xiao Liu (THU) et al. arXiv. [paper] [code] [project page]
• [Aug 2023] "TPTU: Task Planning and Tool Usage of Large Language Model-based AI Agents." Jingqing Ruan (SenseTime) et al. arXiv. [paper]
• 📖 [June 2023] "ToolQA: A Dataset for LLM Question Answering with External Tools." Yuchen Zhuang (GaTech) et al. NeurIPS 2023. [paper] [code]

Survey & Tutorial

• [Sep 2023] "Natural Language based Context Modeling and Reasoning with LLMs: A Tutorial." Haoyi Xiong (Baidu) et al. arXiv. [paper]
• [Sep 2023] "An In-depth Survey of Large Language Model-based Artificial Intelligence Agents." Pengyu Zhao (BJTU) et al. arXiv. [paper]
• 🔥 [Sep 2023] "The Rise and Potential of Large Language Model Based Agents: A Survey." Zhiheng Xi (FDU) et al. arXiv. [paper] [GitHub]
• 🔥 [Aug 2023] "A Survey on Large Language Model based Autonomous Agents." Lei Wang (RUC) et al. arXiv. [paper] [GitHub]
• 🔥 [Mar 2023] "A Survey of Large Language Models (Sec. 6.3 - Planning for Complex Task Solving)." Wayne Xin Zhao (RUC) et al. arXiv. [paper] [GitHub]

Open-Source Projects

Autonomous Task Solver Projects

• Auto-GPT - An experimental open-source attempt to make GPT-4 fully autonomous.
• 🦜️🔗 LangChain - Building applications with LLMs through composability.
• GPT Engineer - Specify what you want it to build, the AI asks for clarification, and then builds it.
• MetaGPT - 🌟 The Multi-Agent Framework: Given one line Requirement, return PRD, Design, Tasks, Repo.
• BabyAGI - An AI-powered task management system.
• L2MAC - 🚀 The LLM Automatic Computer Framework: L2MAC
• Yacana - 🔭🦙 Powering opensource LLMs with multi-agent chats and builing workflows.

Multi-Agent Simulation Projects

• AI Town 🏠💻💌 - A deployable starter kit for building and customizing your own version of AI town - a virtual town where AI characters live, chat and socialize.
• GPTeam - An open-source multi-agent simulation.
• 🏟 ChatArena - Multi-agent language game environments for LLMs.
• 🤖 AgentVerse 🪐 - A flexible framework that simplifies the process of building custom multi-agent environments for large language models (LLMs).

Perspectives

• Language agents: a critical evolutionary step of artificial intelligence - Yu Su (OSU), Sep 5, 2023.
• Introducing XLang: An Open-Source Framework for Building Language Model Agents via Executable Language Grounding - XLANG Lab, Aug 9, 2023.
• What are GPT Agents? A deep dive into the AI interface of the future - Learn why Agents are a core part of the future of AI, Logan Kilpatrick (OpenAI), Jul 25, 2023.
• Language Agents in the Digital World: Opportunities and Risks - Shunyu Yao (Princeton) et al., Jul 24, 2023.
• KokoMind: Can LLMs Understand Social Interactions? - Imagine an AI 🤖 at a cocktail party 🍻, Weiyan Shi (Columbia) et al., Jul, 2023
• LLM Powered Autonomous Agents - Amazing blog by Lilian Weng (OpenAI), Jun 23, 2023.

Other Related Sources

• Personalized Generative AI @ CIKM'23
• LLM-Agents-Papers - A repo lists papers about LLM role playing, memory mechanism and LLM game playing.
• LLMAgentPapers - Must-read papers on multiagents of LLMs.
• awesome-llm-agents - A curated list of awesome LLM agents.

Acknowledgement

We greatly appreciate any contributions via PRs, issues, emails, or other methods. Thanks Tianle Cai (@ctlllll), Yifan Song (@Yifan-Song793), Xinya Du (@xinyadu), Binfeng Xu (@billxbf), Xuanhe Zhou (@zhouxh19), Boyuan Zheng (@boyuanzheng010), Qiao Jin (@Andy-jqa), Shenao Zhang (@shenao-zhang), Yu Gu (@entslscheia), Zhibin Gou (@ZubinGou), Fan Zhou (@koalazf99), Ziniu Hu (@acbull), Yangjun Ruan (@ryoungj), Zhiyuan Hu (@zhiyuanhubj), Qinlin Zhao (@icecream-and-tea), Lucio La Cava (@luciolcv), Zehui Chen (@zehuichen123), Rishi Hazra (@RishiHazra), Lin Guan (@GuanSuns), Yuchen Zhuang (@night-chen), Xuhui Zhou (@XuhuiZhou), Samuel Holt (@samholt) and many others for their kind suggestions and contributions. ❤️

The repository is initially built and maintained by Yupeng Hou (yphou@ucsd.edu).