search

CsabaConsulting / InspectorGadgetApp

Open Multi-Modal Personal Assistant
MIT License
3 stars 1 forks source link

Gemini 1.5 Pro results in 500 Internal error with as many tools as Flash #12

Open MrCsabaToth opened 1 month ago

MrCsabaToth commented 1 month ago

Even though I disabled some tools for gemini-1.5-flash (the Exchange Tool and either the Web Search (Duck Duck Go) or the Tavily Web research tool, and also the Alpha Vantage), but Flash was working with several tools. It seems Pro cannot take as many, or we need to tune some function descriptions? The 500 error happens during the model initialization.

MrCsabaToth commented 1 month ago

For MVP sprint we can neglect this IMHO and roll with the Flash. We can figure it out after the hackathon deadline.

MrCsabaToth commented 1 month ago

There's an idea to stuff the prompt with the list of the available tools and functions. Even though passing the available tools in a native way should give the information to the model, but maybe reinforcing that with a repeated own version would help. Joan Boronat Ruiz advised that and I'll try it https://medium.com/@tocsa2/i-dont-think-it-s-a-good-idea-to-repeat-the-tools-in-the-system-prompt-because-the-model-receives-e90a7df72859

MrCsabaToth commented 1 month ago

I'm using the extra function stuffing. It didn't prevent some internal errors, however it seems to not interfere with the native tool list because I was able to make queries which require two rounds of function calls (weather prompt uses the location function first and then the weather function).