[BUG] Tool inputs being fed through as an array instead of dictionary

[adilazmoon]

Description

Peculiar bug which started occurring when updating from 0.51.0 to 0.63.1. Tools are occasionally being fed arrays of dictionaries instead of dictionary objects. Seems to happen when the agent is expecting to run the tool multiple times for different parts of the task. Double checked, doesn't happen on 0.51.0

Steps to Reproduce

1. Give the agent a task to read 2 files, reddit_ouput.json and news_output.json
2. it reads the first file correctly, then attempts the following input on subsequent tool run: ` Agent: Data Analyst

Using tool: Read a file's content Tool Input: "{\"file_path\": \"reddit_output.json\"}" Tool Output: { "articles": [ { "title": "Pokémon Scarlet and Violet DLC The Hidden Treasure of Area Zero Announced", "url": "https://www.polygon.com/2023/10/15/pokemon-scarlet-violet-dlc-trailer-release-date", "snippet": "Pokémon Scarlet and Violet DLC The Hidden Treasure of Area Zero comes with a new story..." }, { "title": "Pokémon GO: Return of the Halloween Event 2023", "url": "https://www.gamerant.com/pokemon-go-halloween-event-2023/", "snippet": "The Halloween event in Pokémon GO returns with special bonuses and challenges..." }, { "title": "The Pokémon Company Teases New Game Announcements", "url": "https://www.pcgamer.com/pokemon-company-new-game-announcements/", "snippet": "The Pokémon Company has hinted at new upcoming game announcements and updates..." }, { "title": "Pokémon TCG: Sword and Shield Series Coming to an End", "url": "https://www.dexerto.com/pokemon/pokemon-tcg-sword-and-shield-series-coming-to-an-end-2023-2038614/", "snippet": "The Sword and Shield series of Pokémon TCG will conclude by the end of 2023..." }, { "title": "Pokémon Trading Card Game Classic Announced", "url": "https://www.ign.com/articles/pokemon-card-game-classic-announced", "snippet": "The Pokémon Company has officially announced the Pokémon Trading Card Game Classic..." }, { "title": "How to Catch Legendary Pokémon in Pokémon GO", "url": "https://www.gamepur.com/guides/how-to-catch-legendary-pokemon-in-pokemon-go", "snippet": "A guide on catching legendary Pokémon in Pokémon GO..." }, { "title": "The Rise of Pokémon Games on Mobile", "url": "https://www.techradar.com/news/the-rise-of-pokemon-games-on-mobile", "snippet": "An in-depth look at the growing trend of Pokémon games on mobile devices..." }, { "title": "Pokémon: The 10 Most Powerful Pokémon in the TCG", "url": "https://www.sbs.com.au/guide/article/pokemon-the-10-most-powerful-pokemon-in-the-tcg/owsegda1", "snippet": "Ranking the ten most powerful Pokémon in the Pokémon Trading Card Game..." }, { "title": "Pokémon Sleep: Tips and Tricks to Get the Most Out of Your Experience", "url": "https://www.polygon.com/2023/10/14/pokemon-sleep-tips-tricks", "snippet": "Tips and tricks for maximizing experience in Pokémon Sleep..." }, { "title": "Pokémon Legends Arceus: Tips to Level Up Fast", "url": "https://www.techtimes.com/articles/295677/20231014/pokemon-legends-arceus-tips-to-level-up-fast.htm", "snippet": "Strategies to level up quickly in Pokémon Legends Arceus..." } ] }

Agent: Data Analyst

Using tool: Read a file's content Tool Input: "[{\"file_path\": \"news_output.json\"}, {\"articles\": [{\"title\": \"Pok\u00e9mon Trading Card Game Pocket Gets New Gameplay Trailer, Out October\", \"url\": \"https://www.nintendolife.com/news/2024/09/video-pokemon-trading-card-game-pocket-gets-new-gameplay-trailer-out-october\", \"date\": \"Fri 20th Sep 2024\", \"source\": \"Nintendo Life\", \"snippet\": \"Following a release date announcement at the World Championships in August, The Pok\u00e9mon Company has now released a gameplay video for its upcoming mobile title Pok\u00e9mon Trading Card Game Pocket...\"}, {\"title\": \"Pok\u00e9mon GO Confirms Debut of New Dynamax Pokemon\", \"url\": \"https://gamerant.com/pokemon-go-dynamax-falinks-scorbunny-sobble-grookey-debut/\", \"date\": \"3 hours ago\", \"source\": \"Game Rant\", \"snippet\": \"Pokemon GO announces the addition of new Dynamax Pokemon, and they will be available for trainers to capture within the next few days.\"}, {\"title\": \"Pokemon GO Bug Makes It Impossible To Catch One of The Rarest Pokemon\", \"url\": \"https://gamerant.com/pokemon-go-phd-pikachu-impossible-catch-bug/\", \"date\": \"1 day ago\", \"source\": \"Game Rant\", \"snippet\": \"A Pokemon GO bug prevents users from catching one of the rarest Pokemon in the game and fans are hoping that it will be addressed in the next update.\"}, {\"title\": \"Nintendo sues \u2018Pokemon with guns\u2019 game Palworld\", \"url\": \"https://www.independent.co.uk/tech/nintendo-pokemon-palworld-lawsuit-guns-b2615586.html\", \"date\": \"5 days ago\", \"source\": \"Independent\", \"snippet\": \"The Japanese company alleges that the Palworld video game 'infringes multiple patent rights'.\"}, {\"title\": \"New Pok\u00e9mon Animated Short Is Seriously Adorable\", \"url\": \"https://www.nintendolife.com/news/2024/09/random-new-pokemon-animated-short-is-seriously-adorable\", \"date\": \"Tue 17th Sep 2024\", \"source\": \"Nintendo Life\", \"snippet\": \"There\u2019s no shortage of adorable animated Pok\u00e9mon content out there, but The Pok\u00e9mon Company\u2019s latest effort, \u2018Chasing the Moon,\u2019 is one of the best-looking ones we\u2019ve seen in a while.\"}]}]" Tool Output: Error: the Action Input is not a valid key, value dictionary.` Seems to consistently happen on tasks which require a tool to be called multiple times.

Expected behavior

Previously it could intelligently run the tool multiple separate times (in sequence).

Screenshots/Code snippets

Agent: Data Analyst

Task: 1. URL Processor and Web Scraper

Reads the JSON file reddit_output.json and news_output.json. Extracts URLs for each article

Agent: Data Analyst

Using tool: Read a file's content

Tool Input:

"{\"file_path\": \"reddit_output.json\"}"

Tool Output:

{ "articles": [ { "title": "Pokémon Scarlet and Violet DLC The Hidden Treasure of Area Zero Announced", "url": "https://www.polygon.com/2023/10/15/pokemon-scarlet-violet-dlc-trailer-release-date", "snippet": "Pokémon Scarlet and Violet DLC The Hidden Treasure of Area Zero comes with a new story..." }, { "title": "Pokémon GO: Return of the Halloween Event 2023", "url": "https://www.gamerant.com/pokemon-go-halloween-event-2023/", "snippet": "The Halloween event in Pokémon GO returns with special bonuses and challenges..." }, { "title": "The Pokémon Company Teases New Game Announcements", "url": "https://www.pcgamer.com/pokemon-company-new-game-announcements/", "snippet": "The Pokémon Company has hinted at new upcoming game announcements and updates..." }, { "title": "Pokémon TCG: Sword and Shield Series Coming to an End", "url": "https://www.dexerto.com/pokemon/pokemon-tcg-sword-and-shield-series-coming-to-an-end-2023-2038614/", "snippet": "The Sword and Shield series of Pokémon TCG will conclude by the end of 2023..." }, { "title": "Pokémon Trading Card Game Classic Announced", "url": "https://www.ign.com/articles/pokemon-card-game-classic-announced", "snippet": "The Pokémon Company has officially announced the Pokémon Trading Card Game Classic..." }, { "title": "How to Catch Legendary Pokémon in Pokémon GO", "url": "https://www.gamepur.com/guides/how-to-catch-legendary-pokemon-in-pokemon-go", "snippet": "A guide on catching legendary Pokémon in Pokémon GO..." }, { "title": "The Rise of Pokémon Games on Mobile", "url": "https://www.techradar.com/news/the-rise-of-pokemon-games-on-mobile", "snippet": "An in-depth look at the growing trend of Pokémon games on mobile devices..." }, { "title": "Pokémon: The 10 Most Powerful Pokémon in the TCG", "url": "https://www.sbs.com.au/guide/article/pokemon-the-10-most-powerful-pokemon-in-the-tcg/owsegda1", "snippet": "Ranking the ten most powerful Pokémon in the Pokémon Trading Card Game..." }, { "title": "Pokémon Sleep: Tips and Tricks to Get the Most Out of Your Experience", "url": "https://www.polygon.com/2023/10/14/pokemon-sleep-tips-tricks", "snippet": "Tips and tricks for maximizing experience in Pokémon Sleep..." }, { "title": "Pokémon Legends Arceus: Tips to Level Up Fast", "url": "https://www.techtimes.com/articles/295677/20231014/pokemon-legends-arceus-tips-to-level-up-fast.htm", "snippet": "Strategies to level up quickly in Pokémon Legends Arceus..." } ] }

Agent: Data Analyst

Using tool: Read a file's content

Tool Input:

"[{\"file_path\": \"news_output.json\"}, {\"articles\": [{\"title\": \"Pok\u00e9mon Trading Card Game Pocket Gets New Gameplay Trailer, Out October\", \"url\": \"https://www.nintendolife.com/news/2024/09/video-pokemon-trading-card-game-pocket-gets-new-gameplay-trailer-out-october\", \"date\": \"Fri 20th Sep 2024\", \"source\": \"Nintendo Life\", \"snippet\": \"Following a release date announcement at the World Championships in August, The Pok\u00e9mon Company has now released a gameplay video for its upcoming mobile title Pok\u00e9mon Trading Card Game Pocket...\"}, {\"title\": \"Pok\u00e9mon GO Confirms Debut of New Dynamax Pokemon\", \"url\": \"https://gamerant.com/pokemon-go-dynamax-falinks-scorbunny-sobble-grookey-debut/\", \"date\": \"3 hours ago\", \"source\": \"Game Rant\", \"snippet\": \"Pokemon GO announces the addition of new Dynamax Pokemon, and they will be available for trainers to capture within the next few days.\"}, {\"title\": \"Pokemon GO Bug Makes It Impossible To Catch One of The Rarest Pokemon\", \"url\": \"https://gamerant.com/pokemon-go-phd-pikachu-impossible-catch-bug/\", \"date\": \"1 day ago\", \"source\": \"Game Rant\", \"snippet\": \"A Pokemon GO bug prevents users from catching one of the rarest Pokemon in the game and fans are hoping that it will be addressed in the next update.\"}, {\"title\": \"Nintendo sues \u2018Pokemon with guns\u2019 game Palworld\", \"url\": \"https://www.independent.co.uk/tech/nintendo-pokemon-palworld-lawsuit-guns-b2615586.html\", \"date\": \"5 days ago\", \"source\": \"Independent\", \"snippet\": \"The Japanese company alleges that the Palworld video game 'infringes multiple patent rights'.\"}, {\"title\": \"New Pok\u00e9mon Animated Short Is Seriously Adorable\", \"url\": \"https://www.nintendolife.com/news/2024/09/random-new-pokemon-animated-short-is-seriously-adorable\", \"date\": \"Tue 17th Sep 2024\", \"source\": \"Nintendo Life\", \"snippet\": \"There\u2019s no shortage of adorable animated Pok\u00e9mon content out there, but The Pok\u00e9mon Company\u2019s latest effort, \u2018Chasing the Moon,\u2019 is one of the best-looking ones we\u2019ve seen in a while.\"}]}]"

Tool Output:

Error: the Action Input is not a valid key, value dictionary.

Operating System

Windows 11

Python Version

3.12

crewAI Version

0.63.1

crewAI Tools Version

0.12.1

Virtual Environment

Poetry

Evidence

Given above

Possible Solution

Was there a change in how the tools are called in recent updates? I see some changes in tool_usage.py, but nothing I can see would affect it this way.

Additional context

NA

[adilazmoon]

Seems to consistently occur when a tool is executed multiple times in a row. It looks like it appends the previous action output to an array containing the next action input and feeds that in as an input for the next execution

[joaomdmoura]

Looking into it @adilazmoon what specific model are you using? I think this could be the new check for funciton calling support

[joaomdmoura]

Was able to replicate the error, looking into it

[joaomdmoura]

seems to be adhoc, I got the issue once and not again, so probably the model getting confused, I'll test further just in case

[adilazmoon]

@joaomdmoura using OpenAI gpt-4o-mini. Yea it's also intermittent on my end, but definitely noticeable enough that it affects my output quite significantly and consistently.

[adilazmoon]

Okay the more correct thing for me to say is: I don't actually declare any model. Which I assume defaults to OpenAI gpt-4o-mini

[joaomdmoura]

I have an idea to make it better, will try it out. And yes the default is GPT-4o-mini 😊

[ANPCI]

@joaomdmoura I see the issue with Google Gemini-pro , just wanted to know if there is any update on the fix ?

Also, yes this is happening when a tool is executed multiple times in a row,

[ANPCI]

Okay the more correct thing for me to say is: I don't actually declare any model. Which I assume defaults to OpenAI gpt-4o-mini

Hi @adilazmoon , is it working for you now ?

[adilazmoon]

@ANPCI To be perfectly candid, I was sticking to 0.53.1 until this issue was resolved (ran some tests on the early 60 versions, but was still getting the issue). Decided to try and do a few test runs on the latest version 0.67.1 because you tagged me here. It actually looks like it's not an issue anymore. I'll do a few more runs just to be sure.

@joaomdmoura was there anything explicitly done on your side to fix this? Just curious about what the root cause was

[luisandino]

Hi, I am getting the same error. I made two versions of the same function, one receiving four parameters while the second is a dictionary, but I still get the error. The function worked well with the dictionary with an older library version, but now I am using v0.74.2, and it is failing. Are there any instructions on how to overcome this error message?

[sorin-costea]

Possibly the same issue as reported in https://github.com/crewAIInc/crewAI/issues/1522

[nate-walter]

I'm still seeing this bug today

[luisandino]

Hi, I am getting the same error. I made two versions of the same function, one receiving four parameters while the second is a dictionary, but I still get the error. The function worked well with the dictionary with an older library version, but now I am using v0.74.2, and it is failing. Are there any instructions on how to overcome this error message?

I have installed version 0.76.9, and it is working better now. The manager agent correctly calls the corresponding agent, task, and tool. However, sometimes, the agent controlling the tool responds by itself using the pre-trained data from the llm and won't call the tool, which is wrong. I want the agent to always call the tool. I will keep testing and post another update afterwards.

[sorin-costea]

@luisandino happens to me too, more as I increase the temperature. Setting it to 0 it works fine.

[luisandino]

@luisandino happens to me too, more as I increase the temperature. Setting it to 0 it works fine.

@sorin-costea Yes, it worked better by reducing the temperature to zero, still got a different response once after eight tests. I need to keep testing with different parameters and templates for the agent because I must ensure it calls the tool and not trying to infer the answer by itself.

We have to be very cautious with library updates, it can mess up what was working well before.

[hboregio]

Also happening with Composio's official crewai & google calendar example.

Got it to work by changing llm settings to: ChatOpenAI(model="gpt-4o-mini", temperature=0.7)

[jamesdhope]

Same with meta/llama3-70b when tool is called once or several times. CrewAI==0.80.0.

Tool Output:

Error: the Action Input is not a valid key, value dictionary.

[paarttipaabhalaji]

Same with meta/llama3-70b when tool is called once or several times. CrewAI==0.80.0.

Tool Output:

Error: the Action Input is not a valid key, value dictionary.

same for me with watsonx/mistralai/mistral-large model, crewAI==0.80.0

please resolve it quickly.