Closed krrishdholakia closed 3 months ago
picking this up today. got pushed back due to refactoring work over the weekend.
@krrishdholakia thanks. Not sure if you know this, but Google Gemini is capable of automatically calling the function for you.
In the list of tools, you can pass an instance of CallableFunctionDeclaration
, for example:
) long_term_memory_query_function = CallableFunctionDeclaration( name="long_term_memory_query_function", function=self.functions["long_term_memory_query_function"], description="Query assistant long-term memory to remember and recall data. Indicated to retrieve personal and contextual information.", parameters={ "type": "object", "properties": { "query": { "type": "string", "description": "The query string" }, "document_id": { "type": "string", "description": "The id of the entry in the long-term memory. Only provide the
document_id
if you are sure about the variable name." } }, "required": ["query"] }
In this example, self.functions["long_term_memory_query_function"]
points to method with the same signature as described in the schema.
Then, when you start the chat, you can pass something like this:
model.start_chat(enable_automatic_function_calling=True,
Please let us know your thoughts
that looks pretty similar to how openai's function calling works right @guiramos with tool_choice="auto"
btw - this should already be live in v1.40.16
.
https://docs.litellm.ai/docs/providers/gemini#tool-calling
rom litellm import completion
import os
# set env
os.environ["GEMINI_API_KEY"] = ".."
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
},
}
]
messages = [{"role": "user", "content": "What's the weather like in Boston today?"}]
response = completion(
model="gemini/gemini-1.5-flash",
messages=messages,
tools=tools,
tool_choice="auto"
)
@krrishdholakia well, not reallly.
Google gemini will actually make the call to your function, openai api won't do that. You have to parse the response, build the call to the function yourself.
that's interesting - can you point me to docs on this @guiramos i'll see how to add it
we can probably create a separate issue for tracking that
@krrishdholakia Even when I was researching myself I was not able to find the official documentation for this but I can tell you it works. This is what I have:
list_conversations_function = CallableFunctionDeclaration(
name="list_conversations_function",
function=self.functions["list_conversations_function"],
description="List all conversations available with the user.",
parameters={
"type": "object",
"properties": {}
}
)
select_conversation_function = CallableFunctionDeclaration( name="select_conversation_function",
function=self.functions["select_conversation_function"],
description="Select the conversation by its title to be the active one.",
parameters={
"type": "object",
"properties": {
"conversation_title": {
"type": "string",
"description": "The title of the conversation to select"
}
},
"required": ["conversation_title"]
}
)
agent_tool = Tool(
function_declarations=[
list_conversations_function,
select_conversation_function,
]
)
function_calling_config = FunctionCallingConfig()
function_calling_config.mode = FunctionCallingConfig.Mode.ANY
function_calling_config.allowed_function_names.extend([
"list_conversations_function",
"select_conversation_function"
])
Again, pay attention to the function
attribute which needs to point to the actual method that will get called automatically.
You can use FunctionCallingConfig.Mode.AUTO
too without providing the name of the functions.
self.model = genai.GenerativeModel(
model_name=self.model_name,
system_instruction=self.system_instruction,
tools=self.agent_tool,
tool_config=self.tool_config
)
self.chat = self.model.start_chat(enable_automatic_function_calling=True, history=chat_history)
The enable_automatic_function_calling=True
is necessary for automatic function call.
This is my method I have the custom call for the chat
created by gemini api, for some reason I had to pass the tools and tools_config again. Not only n the constructor for the model (above).
def send_message(self, message_content, json_mode=False, json_schema: Optional[List] = None, temperature=0.0):
return self.chat.send_message(content=convert_message_to_gemini(message_content),
generation_config=build_config(json_mode=json_mode, json_schema=json_schema, temperature=temperature),
safety_settings=self.safety_settings,
tools=self.agent_tool,
tool_config=self.tool_config
)
This send_message
method I plan to replace with your litellm
implementation.
what is self.functions
?
@krrishdholakia
In my implementations is an array of function points that matches the one I am passing to gemini.
functions = {
"list_conversations_function": self.list_conversations_function_handler,
"select_conversation_function": self.select_conversation_function_handler,
}
self.prepare_system_message()
self.chat = GoogleChat(system_instruction=self.system_prompt,
functions=functions
)
The self
in the self.functions
you are asking about is the GoogleChat
instance.
@krrishdholakia
Did you give up?
hi @guiramos, dealing with some p0s.
Would welcome a PR for this.
https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/function-calling#curl_2
tried using this locally with
toolconfig
. This seems like it would only work with later versions of the vertex ai sdk.Considering we should just move to making http calls (like we do for anthropic models) - will allow everyone to use the latest gemini features without needing to upgrade litellm + vertex ai