search

sublayerapp / sublayer

A model-agnostic Ruby Generative AI DSL and framework. Provides base classes for building Generators, Actions, Tasks, and Agents that can be used to build AI powered applications in Ruby.
https://docs.sublayer.com
MIT License
119 stars 2 forks source link

Occasional 500 errors from newer gemini models #46

Closed swerner closed 3 months ago

swerner commented 4 months ago

Started working on adding the list_of_strings output adapter, and along the way updated the gemini provider to use the function calling syntax that should be enabled on the latest models: gemini-1.0-pro gemini-1.0-pro-001 gemini-1.5-flash-latest gemini-1.5-pro-latest as described here: https://ai.google.dev/gemini-api/docs/function-calling

Have to do multiple calls to get it to respond with something other than 500, and occasionally it doesn't actually do a function call and just returns a text response.

AndrewBKang commented 4 months ago

The weirdest thing... the tests pass in sublayer if I simply remove the line: "Take a deep breath and think step by step before you start coding." from the example generator

swerner commented 4 months ago

Oh that's super interesting!

AndrewBKang commented 4 months ago

So I was wondering if it had to do with the sentence itself. Tried it on historical event generator but it's working fine. so it might not be the content of the sentence. I'm wondering if it's the size of the text now. Going to see what the default context window is for gemini. and then play with the size of the text in the prompt.

swerner commented 3 months ago

Still getting issues here across the board even as we add additional output adapters. I'll keep digging to try to find a way to bring gemini back...

swerner commented 3 months ago

spent a bunch of time generating with the new gemini provider, this will be fixed in the 0.2.5 release

swerner commented 3 months ago

Fixed as of 0.2.5