Using gemini-1.5-flash.
Prompt: "What is the weather today?"
Tool configuration 1:
[
Tool(
functionDeclarations: [
FunctionDeclaration(
'fetchWeatherForecast',
'Returns the weather in a given location.',
Schema(
SchemaType.object,
properties: {
'latitude': Schema.number(
description: 'Latitude of the weather observation and forecast',
),
'longitude': Schema.number(
description:
'Longitude of the weather observation and forecast',
),
},
requiredProperties: ['latitude', 'longitude'],
),
),
],
),
Tool(
functionDeclarations: [
FunctionDeclaration(
'fetchGpsLocation',
'Returns the latitude and longitude of the current GPS location.',
Schema(SchemaType.string),
),
FunctionDeclaration(
'fetchHeartRate',
'Returns the current heart rate measurement.',
Schema(SchemaType.integer),
),
],
),
]
Response 1: I need to know your location to get the weather forecast. Can you tell me your latitude and longitude?
Tool configuration 2 (switching the order of the two tools):
[
Tool(
functionDeclarations: [
FunctionDeclaration(
'fetchGpsLocation',
'Returns the latitude and longitude of the current GPS location.',
Schema(SchemaType.string),
),
FunctionDeclaration(
'fetchHeartRate',
'Returns the current heart rate measurement.',
Schema(SchemaType.integer),
),
],
),
Tool(
functionDeclarations: [
FunctionDeclaration(
'fetchWeatherForecast',
'Returns the weather in a given location.',
Schema(
SchemaType.object,
properties: {
'latitude': Schema.number(
description: 'Latitude of the weather observation and forecast',
),
'longitude': Schema.number(
description:
'Longitude of the weather observation and forecast',
),
},
requiredProperties: ['latitude', 'longitude'],
),
),
],
),
]
Response 2:
I am sorry, I cannot fulfill this request. I do not have access to weather information.
Artificially unifying the two tools into a virtual one, merging all functions under that:
[
Tool(
functionDeclarations: [
FunctionDeclaration(
'fetchWeatherForecast',
'Returns the weather in a given location.',
Schema(
SchemaType.object,
properties: {
'latitude': Schema.number(
description: 'Latitude of the weather observation and forecast',
),
'longitude': Schema.number(
description:
'Longitude of the weather observation and forecast',
),
},
requiredProperties: ['latitude', 'longitude'],
),
),
FunctionDeclaration(
'fetchGpsLocation',
'Returns the latitude and longitude of the current GPS location.',
Schema(SchemaType.string),
),
FunctionDeclaration(
'fetchHeartRate',
'Returns the current heart rate measurement.',
Schema(SchemaType.integer),
),
],
),
]
The model properly asks first for the location tool for the lat / lon coordinates, and then with a second round of call (where I pass the location function call result down) it properly asks to call the weather function with the proper GPS coordinates.
Respnose: The weather today is clear and the temperature is 25 degrees Celsius. The wind is blowing from the southwest at 2 meters per second.
Actual vs expected behavior:
I'd expect to be able to keep the two tools apart for extensible software architecture. So far I'm aiming for 8 tools, many of those have multiple functions.
Description of the bug:
Using
gemini-1.5-flash
. Prompt: "What is the weather today?"Tool configuration 1:
Response 1:
I need to know your location to get the weather forecast. Can you tell me your latitude and longitude?
Tool configuration 2 (switching the order of the two tools):
Response 2:
I am sorry, I cannot fulfill this request. I do not have access to weather information.
Artificially unifying the two tools into a virtual one, merging all functions under that:
The model properly asks first for the location tool for the lat / lon coordinates, and then with a second round of call (where I pass the location function call result down) it properly asks to call the weather function with the proper GPS coordinates.
Respnose:
The weather today is clear and the temperature is 25 degrees Celsius. The wind is blowing from the southwest at 2 meters per second.
Actual vs expected behavior:
I'd expect to be able to keep the two tools apart for extensible software architecture. So far I'm aiming for 8 tools, many of those have multiple functions.
Any other information you'd like to share?
I peeked at the code and it looks to me that ultimately a REST API call is made. The
_generateContentRequest
seems to serialize the tools OK (by looking at it)if (tools != null) 'tools': tools.map((t) => t.toJson()).toList(),
https://github.com/google-gemini/generative-ai-dart/blob/8a58d778e2ef254901e3319f238b147788211ecf/pkgs/google_generative_ai/lib/src/model.dart#L333 so I don't know yet where's the problem. Maybe in my code?