joshfree
commented
6 months ago @mssfang please take a look
Open sravanthi-tl opened 6 months ago
joshfree
commented
6 months ago @mssfang please take a look
mssfang
commented
5 months ago Hi, @sravanthi-tl Thank you for letting us know about this.
I have tested by using OpenAI cookbook parallel tool call sample: https://platform.openai.com/docs/guides/function-calling/parallel-function-calling
Find if I use "gpt-35-turbo-1106", a simple one response returned which is
Final Result: The weather in San Francisco, Tokyo, and Paris is -7 degrees Celsius.However, if I use gpt-4-turbo, all three cities' weather are returned.
- **San Francisco:** The temperature is around 12°C with clear skies.
- **Tokyo:** It is approximately 5°C with cloudy conditions.
- **Paris:** The temperature is about -7°C with snow falling.
Please note, the weather can change rapidly, so it’s good to check a reliable source if you need real-time updates.The sample code pasted in below:
public class ParallelToolCalls {
public static void main(String[] args) {
String azureOpenaiKey = Configuration.getGlobalConfiguration().get("AZURE_OPENAI_KEY");
String endpoint = Configuration.getGlobalConfiguration().get("AZURE_OPENAI_ENDPOINT");
String deploymentOrModelId = "gpt-35-turbo-1106";
// String deploymentOrModelId = "gpt-4-turbo";
OpenAIClient client = new OpenAIClientBuilder()
.serviceVersion(OpenAIServiceVersion.V2024_02_15_PREVIEW)
.endpoint(endpoint)
.credential(new AzureKeyCredential(azureOpenaiKey))
.buildClient();
List<ChatRequestMessage> chatMessages = Arrays.asList(
new ChatRequestSystemMessage("You are a helpful assistant."),
new ChatRequestUserMessage("What's the weather like in San Francisco, Tokyo, and Paris?")
);
ChatCompletionsToolDefinition toolDefinition = new ChatCompletionsFunctionToolDefinition(
getCurrentWeather());
ChatCompletionsOptions chatCompletionsOptions = new ChatCompletionsOptions(chatMessages)
.setTemperature(0.0)
.setTools(Arrays.asList(toolDefinition))
.setToolChoice(BinaryData.fromObject("auto"));
IterableStream<ChatCompletions> chatCompletionsStream = client.getChatCompletionsStream(deploymentOrModelId,
chatCompletionsOptions);
String toolCallId = null;
String functionName = null;
StringBuilder functionArguments = new StringBuilder();
CompletionsFinishReason finishReason = null;
for (ChatCompletions chatCompletions : chatCompletionsStream) {
// In the case of Azure, the 1st message will contain filter information but no choices sometimes
if (chatCompletions.getChoices().isEmpty()) {
continue;
}
ChatChoice choice = chatCompletions.getChoices().get(0);
if (choice.getFinishReason() != null) {
finishReason = choice.getFinishReason();
}
List<ChatCompletionsToolCall> toolCalls = choice.getDelta().getToolCalls();
// We take the functionName when it's available, and we aggregate the arguments.
// We also monitor FinishReason for TOOL_CALL. That's the LLM signaling we should
// call our function
if (toolCalls != null) {
ChatCompletionsFunctionToolCall toolCall = (ChatCompletionsFunctionToolCall) toolCalls.get(0);
if (toolCall != null) {
functionArguments.append(toolCall.getFunction().getArguments());
if (toolCall.getId() != null) {
toolCallId = toolCall.getId();
}
if (toolCall.getFunction().getName() != null) {
functionName = toolCall.getFunction().getName();
}
}
}
}
System.out.println("Tool Call Id: " + toolCallId);
System.out.println("Function Name: " + functionName);
System.out.println("Function Arguments: " + functionArguments);
System.out.println("Finish Reason: " + finishReason);
// We verify that the LLM wants us to call the function we advertised in the original request
// Preparation for follow-up with the service we add:
// - All the messages we sent
// - The ChatCompletionsFunctionToolCall from the service as part of a ChatRequestAssistantMessage
// - The result of function tool as part of a ChatRequestToolMessage
if (finishReason == CompletionsFinishReason.TOOL_CALLS) {
// Here the "content" can be null if used in non-Azure OpenAI
// We prepare the assistant message reminding the LLM of the context of this request. We provide:
// - The tool call id
// - The function description
FunctionCall functionCall = new FunctionCall(functionName, functionArguments.toString());
ChatCompletionsFunctionToolCall functionToolCall = new ChatCompletionsFunctionToolCall(toolCallId, functionCall);
ChatRequestAssistantMessage assistantRequestMessage = new ChatRequestAssistantMessage("");
assistantRequestMessage.setToolCalls(Arrays.asList(functionToolCall));
// As an additional step, you may want to deserialize the parameters, so you can call your function
FunctionArguments parameters = BinaryData.fromString(functionArguments.toString()).toObject(FunctionArguments.class);
System.out.println("Location Name: " + parameters.locationName);
System.out.println("Date: " + parameters.date);
String functionCallResult = futureTemperature(parameters.locationName, parameters.date);
// This message contains the information that will allow the LLM to resume the text generation
ChatRequestToolMessage toolRequestMessage = new ChatRequestToolMessage(functionCallResult, toolCallId);
List<ChatRequestMessage> followUpMessages = Arrays.asList(
// We add the original messages from the request
chatMessages.get(0),
chatMessages.get(1),
assistantRequestMessage,
toolRequestMessage
);
IterableStream<ChatCompletions> followUpChatCompletionsStream = client.getChatCompletionsStream(
deploymentOrModelId, new ChatCompletionsOptions(followUpMessages));
StringBuilder finalResult = new StringBuilder();
CompletionsFinishReason finalFinishReason = null;
for (ChatCompletions chatCompletions : followUpChatCompletionsStream) {
if (chatCompletions.getChoices().isEmpty()) {
continue;
}
ChatChoice choice = chatCompletions.getChoices().get(0);
if (choice.getFinishReason() != null) {
finalFinishReason = choice.getFinishReason();
}
if (choice.getDelta().getContent() != null) {
finalResult.append(choice.getDelta().getContent());
}
}
// We verify that the LLM has STOPPED as a finishing reason
if (finalFinishReason == CompletionsFinishReason.STOPPED) {
System.out.println("Final Result: " + finalResult);
}
}
}
private static Map<String, String> getCurrentWeather(String location) {
String unit = "fahrenheit";
// Get the current weather in a given location
if ("tokyo".equalsIgnoreCase(location)) {
Map<String, String> res = new HashMap<>();
res.put("location", "Tokyo");
res.put("temperature", "10");
res.put("unit", unit);
return res;
}
else if ("san francisco".equalsIgnoreCase(location)) {
Map<String, String> res = new HashMap<>();
res.put("location", "San Francisco");
res.put("temperature", "72");
res.put("unit", unit);
return res;
}
else if ("paris".equalsIgnoreCase(location)) {
Map<String, String> res = new HashMap<>();
res.put("location", "Paris");
res.put("temperature", "22");
res.put("unit", unit);
return res;
}
else {
Map<String, String> res = new HashMap<>();
res.put("location", location);
res.put("temperature", "unknown");
return res;
}
}
// In this example we ignore the parameters for our tool function
private static String futureTemperature(String locationName, String data) {
return "-7 C";
}
private static FunctionDefinition getCurrentWeather() {
FunctionDefinition functionDefinition = new FunctionDefinition("GetCurrentWeather");
functionDefinition.setDescription("Get the current weather for a given location.");
FunctionArguments parameters = new FunctionArguments();
functionDefinition.setParameters(BinaryData.fromObject(parameters));
return functionDefinition;
}
private static FunctionDefinition getFutureTemperatureFunctionDefinition() {
FunctionDefinition functionDefinition = new FunctionDefinition("FutureTemperature");
functionDefinition.setDescription("Get the future temperature for a given location and date.");
FutureTemperatureParameters parameters = new FutureTemperatureParameters();
functionDefinition.setParameters(BinaryData.fromObject(parameters));
return functionDefinition;
}
private static class FunctionArguments {
@JsonProperty(value = "location_name")
private String locationName;
@JsonProperty(value = "date")
private String date;
}
private static class FutureTemperatureParameters {
@JsonProperty(value = "type")
private String type = "object";
@JsonProperty(value = "properties")
private FutureTemperatureProperties properties = new FutureTemperatureProperties();
}
private static class FutureTemperatureProperties {
@JsonProperty(value = "unit") StringField unit = new StringField("Temperature unit. Can be either Celsius or Fahrenheit. Defaults to Celsius.");
@JsonProperty(value = "location_name") StringField locationName = new StringField("The name of the location to get the future temperature for.");
@JsonProperty(value = "date") StringField date = new StringField("The date to get the future temperature for. The format is YYYY-MM-DD.");
}
private static class StringField {
@JsonProperty(value = "type")
private final String type = "string";
@JsonProperty(value = "description")
private String description;
@JsonCreator
StringField(@JsonProperty(value = "description") String description) {
this.description = description;
}
}
}Which made me wondering what is your sample looks like?
jayshetti
commented
5 months ago Hi, @sravanthi-tl Thank you for letting us know about this.
I have tested by using OpenAI cookbook parallel tool call sample: https://platform.openai.com/docs/guides/function-calling/parallel-function-calling
Find if I use "gpt-35-turbo-1106", a simple one response returned which is
Final Result: The weather in San Francisco, Tokyo, and Paris is -7 degrees Celsius.However, if I use
gpt-4-turbo, all three cities' weather are returned.- **San Francisco:** The temperature is around 12°C with clear skies. - **Tokyo:** It is approximately 5°C with cloudy conditions. - **Paris:** The temperature is about -7°C with snow falling. Please note, the weather can change rapidly, so it’s good to check a reliable source if you need real-time updates.The sample code pasted in below:
public class ParallelToolCalls { public static void main(String[] args) { String azureOpenaiKey = Configuration.getGlobalConfiguration().get("AZURE_OPENAI_KEY"); String endpoint = Configuration.getGlobalConfiguration().get("AZURE_OPENAI_ENDPOINT"); String deploymentOrModelId = "gpt-35-turbo-1106"; // String deploymentOrModelId = "gpt-4-turbo"; OpenAIClient client = new OpenAIClientBuilder() .serviceVersion(OpenAIServiceVersion.V2024_02_15_PREVIEW) .endpoint(endpoint) .credential(new AzureKeyCredential(azureOpenaiKey)) .buildClient(); List<ChatRequestMessage> chatMessages = Arrays.asList( new ChatRequestSystemMessage("You are a helpful assistant."), new ChatRequestUserMessage("What's the weather like in San Francisco, Tokyo, and Paris?") ); ChatCompletionsToolDefinition toolDefinition = new ChatCompletionsFunctionToolDefinition( getCurrentWeather()); ChatCompletionsOptions chatCompletionsOptions = new ChatCompletionsOptions(chatMessages) .setTemperature(0.0) .setTools(Arrays.asList(toolDefinition)) .setToolChoice(BinaryData.fromObject("auto")); IterableStream<ChatCompletions> chatCompletionsStream = client.getChatCompletionsStream(deploymentOrModelId, chatCompletionsOptions); String toolCallId = null; String functionName = null; StringBuilder functionArguments = new StringBuilder(); CompletionsFinishReason finishReason = null; for (ChatCompletions chatCompletions : chatCompletionsStream) { // In the case of Azure, the 1st message will contain filter information but no choices sometimes if (chatCompletions.getChoices().isEmpty()) { continue; } ChatChoice choice = chatCompletions.getChoices().get(0); if (choice.getFinishReason() != null) { finishReason = choice.getFinishReason(); } List<ChatCompletionsToolCall> toolCalls = choice.getDelta().getToolCalls(); // We take the functionName when it's available, and we aggregate the arguments. // We also monitor FinishReason for TOOL_CALL. That's the LLM signaling we should // call our function if (toolCalls != null) { ChatCompletionsFunctionToolCall toolCall = (ChatCompletionsFunctionToolCall) toolCalls.get(0); if (toolCall != null) { functionArguments.append(toolCall.getFunction().getArguments()); if (toolCall.getId() != null) { toolCallId = toolCall.getId(); } if (toolCall.getFunction().getName() != null) { functionName = toolCall.getFunction().getName(); } } } } System.out.println("Tool Call Id: " + toolCallId); System.out.println("Function Name: " + functionName); System.out.println("Function Arguments: " + functionArguments); System.out.println("Finish Reason: " + finishReason); // We verify that the LLM wants us to call the function we advertised in the original request // Preparation for follow-up with the service we add: // - All the messages we sent // - The ChatCompletionsFunctionToolCall from the service as part of a ChatRequestAssistantMessage // - The result of function tool as part of a ChatRequestToolMessage if (finishReason == CompletionsFinishReason.TOOL_CALLS) { // Here the "content" can be null if used in non-Azure OpenAI // We prepare the assistant message reminding the LLM of the context of this request. We provide: // - The tool call id // - The function description FunctionCall functionCall = new FunctionCall(functionName, functionArguments.toString()); ChatCompletionsFunctionToolCall functionToolCall = new ChatCompletionsFunctionToolCall(toolCallId, functionCall); ChatRequestAssistantMessage assistantRequestMessage = new ChatRequestAssistantMessage(""); assistantRequestMessage.setToolCalls(Arrays.asList(functionToolCall)); // As an additional step, you may want to deserialize the parameters, so you can call your function FunctionArguments parameters = BinaryData.fromString(functionArguments.toString()).toObject(FunctionArguments.class); System.out.println("Location Name: " + parameters.locationName); System.out.println("Date: " + parameters.date); String functionCallResult = futureTemperature(parameters.locationName, parameters.date); // This message contains the information that will allow the LLM to resume the text generation ChatRequestToolMessage toolRequestMessage = new ChatRequestToolMessage(functionCallResult, toolCallId); List<ChatRequestMessage> followUpMessages = Arrays.asList( // We add the original messages from the request chatMessages.get(0), chatMessages.get(1), assistantRequestMessage, toolRequestMessage ); IterableStream<ChatCompletions> followUpChatCompletionsStream = client.getChatCompletionsStream( deploymentOrModelId, new ChatCompletionsOptions(followUpMessages)); StringBuilder finalResult = new StringBuilder(); CompletionsFinishReason finalFinishReason = null; for (ChatCompletions chatCompletions : followUpChatCompletionsStream) { if (chatCompletions.getChoices().isEmpty()) { continue; } ChatChoice choice = chatCompletions.getChoices().get(0); if (choice.getFinishReason() != null) { finalFinishReason = choice.getFinishReason(); } if (choice.getDelta().getContent() != null) { finalResult.append(choice.getDelta().getContent()); } } // We verify that the LLM has STOPPED as a finishing reason if (finalFinishReason == CompletionsFinishReason.STOPPED) { System.out.println("Final Result: " + finalResult); } } } private static Map<String, String> getCurrentWeather(String location) { String unit = "fahrenheit"; // Get the current weather in a given location if ("tokyo".equalsIgnoreCase(location)) { Map<String, String> res = new HashMap<>(); res.put("location", "Tokyo"); res.put("temperature", "10"); res.put("unit", unit); return res; } else if ("san francisco".equalsIgnoreCase(location)) { Map<String, String> res = new HashMap<>(); res.put("location", "San Francisco"); res.put("temperature", "72"); res.put("unit", unit); return res; } else if ("paris".equalsIgnoreCase(location)) { Map<String, String> res = new HashMap<>(); res.put("location", "Paris"); res.put("temperature", "22"); res.put("unit", unit); return res; } else { Map<String, String> res = new HashMap<>(); res.put("location", location); res.put("temperature", "unknown"); return res; } } // In this example we ignore the parameters for our tool function private static String futureTemperature(String locationName, String data) { return "-7 C"; } private static FunctionDefinition getCurrentWeather() { FunctionDefinition functionDefinition = new FunctionDefinition("GetCurrentWeather"); functionDefinition.setDescription("Get the current weather for a given location."); FunctionArguments parameters = new FunctionArguments(); functionDefinition.setParameters(BinaryData.fromObject(parameters)); return functionDefinition; } private static FunctionDefinition getFutureTemperatureFunctionDefinition() { FunctionDefinition functionDefinition = new FunctionDefinition("FutureTemperature"); functionDefinition.setDescription("Get the future temperature for a given location and date."); FutureTemperatureParameters parameters = new FutureTemperatureParameters(); functionDefinition.setParameters(BinaryData.fromObject(parameters)); return functionDefinition; } private static class FunctionArguments { @JsonProperty(value = "location_name") private String locationName; @JsonProperty(value = "date") private String date; } private static class FutureTemperatureParameters { @JsonProperty(value = "type") private String type = "object"; @JsonProperty(value = "properties") private FutureTemperatureProperties properties = new FutureTemperatureProperties(); } private static class FutureTemperatureProperties { @JsonProperty(value = "unit") StringField unit = new StringField("Temperature unit. Can be either Celsius or Fahrenheit. Defaults to Celsius."); @JsonProperty(value = "location_name") StringField locationName = new StringField("The name of the location to get the future temperature for."); @JsonProperty(value = "date") StringField date = new StringField("The date to get the future temperature for. The format is YYYY-MM-DD."); } private static class StringField { @JsonProperty(value = "type") private final String type = "string"; @JsonProperty(value = "description") private String description; @JsonCreator StringField(@JsonProperty(value = "description") String description) { this.description = description; } } }Which made me wondering what is your sample looks like?
@mssfang I tried your sample, if i pass "What's the weather like in Australia?" as prompt i get empty functionArguments, if i pass your prompt "What's the weather like in San Francisco, Tokyo, and Paris?" i see the functionArguments as below
{"location": "San Francisco"}{"location": "Tokyo"}{"location": "Paris"}
Am i missing something here? why this weird behavior?
Query/Question I have a openai fn defined as tool which makes a function call cancel_booking with all required user details. We are using 2024-02-15-preview model which is deployed on azure. Keeping the tool definition same and model version same along with temperature ... we made calls using
On user_message: Hello this is Barry Spencer. Im personal assistant to the Smith's. Can you please cancel appointments of Will Smith and Jaden Smith.
Am unable to figure out the difference between these 2 calls and why there is a difference in response.
Tool Definition
java call
log stmt of request sent to openai
May 05 21:02:22.514 [ForkJoinPool.commonPool-worker-9] INFO com.azure.ai.openai.implementation.OpenAIClientImpl$OpenAIClientService.getChatCompletionsSync - {"az.sdk.message":"HTTP response","statusCode":200,"url":"https://HOST.openai.azure.com/openai/deployments/gpt-35-turbo-0125/chat/completions?api-version=2024-02-15-preview","durationMs":1663}java call response received
py side log of request made and response received