Closed yamazakikakuyo closed 2 months ago
As far as I can tell in my own testing and usage, the tools
kwarg will only take one Tool
class with 1 or more functions defined. So you'll need to represent your grounding call as another function call and add it to the same Tool
that contains the 4 BQ/SQL functions.
I apologize for the late reply. Perhaps, could you please provide the best practive to build the calling function for grounding call? Anyway, Thank you very much for your insight. It means a lot to me. More or less, I can see how to implement your solution to my application. =D
Got it. Sure thing, let me reopen this and come up with a better way for us to document using the grounding tool alongside functions-as-tools. Thanks for opening this!
Thank you for reopening the issue! Looking forward to the improved documentation on using the grounding tool alongside functions-as-tools.
Hi @yamazakikakuyo. Unfortunately I was not able to find a good way to solve this and use both Tools at the same time, nor was I able to decompose the underlying FunctionDeclarations
and grounding retrieval functions and combine them into a Tool.
The current way of using both Tools is to either 1) make a Gemini API call with the grounding tool, then make subsequent Gemini API calls with a Tool that refers to one or more functions, or 2) craft a new FunctionDeclaration
that you then use to manually invoke a Gemini API call within another function call (a bit redundant and nested, I know).
In the meantime, please open a feature request on the Vertex AI issue tracker since this seems like a solid use case to me of combining different Tool specs or Tools with Gemini API calls.
Hi @koverholt, Thank you for your detailed response and the suggestions provided. I understand the current limitations and the workarounds you mentioned for using both Tools simultaneously.
I have opened feature request on the Vertex AI issue tracker to highlight this use case of combining different Tool specs or Tools with Gemini API calls. I hope I could contribute on Gemini development. Nevertheless, your input has been very helpful, and I appreciate your guidance on this matter.
Best Regards.
Thank you for opening that feature request, we really appreciate it! Linking it here for future reference:
File Name
Using Vertex Search (a.k.a Agent Builder) and Calling Function
What happened?
Hi, I have an issue when using more than one Tool class in GenerativeModel class. Long short story, I tried to make a Gemini chatbot that capable to use Grounding (with Agent Builder) and Callign Function, so Gemini chatbot could maximize its potential. I read the documentation of GenerativeModel class in this link and it said that the tools parameter could take more than one Tool class. Here's the code I used:
list_tables_func = FunctionDeclaration( name="list_tables", description="List tables in a dataset that will help answer the user's question", parameters={ "type": "object", "properties": { "dataset_id": { "type": "string", "description": "Dataset ID to fetch tables from.", } }, "required": [ "dataset_id", ], }, )
get_table_func = FunctionDeclaration( name="get_table", description="Get information about a table, including the description, schema, and number of rows that will help answer the user's question. Always use the fully qualified dataset and table names.", parameters={ "type": "object", "properties": { "table_id": { "type": "string", "description": "Fully qualified ID of the table to get information about", } }, "required": [ "table_id", ], }, )
sql_query_func = FunctionDeclaration( name="sql_query", description="Get information from data in BigQuery using SQL queries", parameters={ "type": "object", "properties": { "query": { "type": "string", "description": "SQL query on a single line that will help give quantitative answers to the user's question when run on a BigQuery dataset and table. In the SQL query, always use the fully qualified dataset and table names.", } }, "required": [ "query", ], }, )
sql_query_tool = Tool( function_declarations=[ list_datasets_func, list_tables_func, get_table_func, sql_query_func, ], )
vertex_search_tool = Tool.from_retrieval( retrieval=preview_generative_models.grounding.Retrieval( source=preview_generative_models.grounding.VertexAISearch( datastore=path+f'/dataStores/{DATASTORE_ID}' ), ) )
model = GenerativeModel( "gemini-1.0-pro", generation_config={"temperature": 0}, tools=[ vertex_search_tool, sql_query_tool, ], )
chat = model.start_chat() client = bigquery.Client(project=PROJECT_ID) prompt = "Get a list of datasets that will help answer the user's question." response = chat.send_message(prompt)
Code of Conduct