search

lgrammel / js-agent

Build AI Agents with JS & TS
MIT License
229 stars 35 forks source link

Agent get stuck in the loop at the end #2

Closed Maxim-Filimonov closed 1 year ago

Maxim-Filimonov commented 1 year ago

Hi @lgrammel, First of all great work on creating a well factored and structured code. I'd love to contribute to it as I feel with increasing complexity of autonomous agents it will make a big difference. If you look at the most popular "Auto-GPT" it's drowning in PRs. It seem its getting harder and harder to manage those for the author as the whole system made an experiment which grew very fast in popularity. I tried testing gptagent.js using the code from README and noticed that I don't get done step at the end. Agent just gets stuck in loop of thinking: https://share.cleanshot.com/9V1sYxZ1

lgrammel commented 1 year ago

Thanks for the bug report. I've seen this with GPT-3.5-turbo and will take a look.

Regarding the code, it's changing a lot still. One area that is already somewhat stable and would need contributions are the tools. For example SERPApi, generic api wrapping, openapi spec, etc. would make great tools.

lgrammel commented 1 year ago

I've fixed the issue (size of input when summarizing with GPT-3.5-turbo) and made a few other improvements to the wikipedia-qa agent (fully recursive summarization, better instructions for gpt-3.5-turbo). It should work in the upcoming v0.0.7 release. Leaving this issue open until v0.0.7 has been released.

Btw, for this specific the example, the New York City wikipedia page is quite large, so recursive summarization takes times and costs a bit of money.

lgrammel commented 1 year ago

Leaving this open (even with v0.0.7) as I'm seeing this behavior with gpt-3.5-turbo from time to time.

Maxim-Filimonov commented 1 year ago

Regarding the code, it's changing a lot still. One area that is already somewhat stable and would need contributions are the tools. For example SERPApi, generic api wrapping, openapi spec, etc. would make great tools.

Can steal copy from langchain probably most of the implementation :)

lgrammel commented 1 year ago

Regarding the code, it's changing a lot still. One area that is already somewhat stable and would need contributions are the tools. For example SERPApi, generic api wrapping, openapi spec, etc. would make great tools.

Can steal copy from langchain probably most of the implementation :)

Sure - I haven't looked much at their code, because I want to develop my own understanding and concepts. Many tools will most likely be fairly trivial wrappers.

One aspect that I want to pay a lot of attention to is error handling. Agents will get tool calls wrong from time to time, and providing good AI-oriented error messages can help them recover quickly in many cases. This will help with agent robustness.

lgrammel commented 1 year ago

I've added FlexibleJsonActionFormat, which should help a lot with GPT-3.5-turbo. GPT-3.5-turbo agents should work a lot better now.