Open edeandrea opened 2 weeks ago
geoand
commented
2 weeks ago Thanks for looking into this, but I am pretty sure this is not the proper solution and likely works by accident (and likely won't work for the next type we attempt to tackle).
Can you paste the request / response to / from OpenAI?
edeandrea
commented
2 weeks ago I cut out the calls to the embedding model - that part is irrelevant.
I think I see what you're saying. The marshaller or whatever doesn't seem to understand how to construct a LocalDate to pass to the tool.
But it did get further than it did before!
Have any thoughts on what the next step might be?
2024-11-06 08:12:55,343 INFO [io.qua.lan.ope.com.OpenAiRestApi$OpenAiClientLogger] (vert.x-eventloop-thread-4) Request:
- method: POST
- url: https://api.openai.com/v1/chat/completions
- headers: [Accept: application/json], [Authorization: Be...1f], [Content-Type: application/json], [User-Agent: langchain4j-openai], [content-length: 3979]
- body: {
"model" : "gpt-4o-mini",
"messages" : [ {
"role" : "system",
"content" : "You are a customer chat support agent of an airline named \"Funnair\".\nRespond in a friendly, helpful, and joyful manner.\nYou are interacting with customers through an online chat system.\nBefore providing information about a booking or cancelling a booking,\nyou MUST ensure you have the following information from the user:\nbooking number, customer first name, and last name.\nCheck the message history for this information before asking the user.\nBefore changing a booking, you MUST ensure it is permitted by the terms.\nIf there is a charge for the change, you MUST ask the user to consent before proceeding.\nUse the provided functions to fetch booking details, change bookings, and cancel bookings.\nToday is 2024-11-06.\n"
}, {
"role" : "user",
"content" : "Hello I'm Robert Taylor and my booking number is 105. I'd like to change my flight date to November 16, 2024.\n\nAnswer using the following information:\n2. Changing Bookings\n- Changes allowed up to 24 hours before flight.\n- Change via online or contact our support.\n- Change fee: $50 for Economy, $30 for Premium Economy, Free for Business Class.\n\n1. Booking Flights\n- Book via our website or mobile app.\n- Full payment required at booking.\n- Ensure accuracy of personal information (Name, ID, etc.) as corrections may incur a $25 fee."
} ],
"temperature" : 0.0,
"top_p" : 1.0,
"presence_penalty" : 0.0,
"frequency_penalty" : 0.0,
"tools" : [ {
"type" : "function",
"function" : {
"name" : "getBookingDetails",
"description" : "Retrieves information about an existing booking,\nsuch as the flight date, booking status, departure and arrival airports, and booking class.\n",
"parameters" : {
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName" ]
}
}
}, {
"type" : "function",
"function" : {
"name" : "changeBooking",
"description" : "Modifies an existing booking.\nThis includes making changes to the flight date, and the departure and arrival airports.\n",
"parameters" : {
"type" : "object",
"properties" : {
"newFlightDate" : {
"type" : "object",
"properties" : {
"month" : {
"type" : "integer"
},
"year" : {
"type" : "integer"
},
"day" : {
"type" : "integer"
}
},
"required" : [ ]
},
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"newDepartureAirport" : {
"type" : "string",
"description" : "3-letter code for departure airport"
},
"newArrivalAirport" : {
"type" : "string",
"description" : "3-letter code for arrival airport"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName", "newFlightDate", "newDepartureAirport", "newArrivalAirport" ]
}
}
}, {
"type" : "function",
"function" : {
"name" : "cancelBooking",
"description" : "",
"parameters" : {
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName" ]
}
}
} ]
}
2024-11-06 08:12:56,230 INFO [io.qua.lan.ope.com.OpenAiRestApi$OpenAiClientLogger] (vert.x-eventloop-thread-4) Response:
- status code: 200
- headers: [Date: Wed, 06 Nov 2024 13:12:56 GMT], [Content-Type: application/json], [Content-Length: 1092], [Connection: keep-alive], [access-control-expose-headers: X-Request-ID], [openai-organization: red-hat-jdskfr], [openai-processing-ms: 615], [openai-version: 2020-10-01], [x-ratelimit-limit-requests: 10000], [x-ratelimit-limit-tokens: 200000], [x-ratelimit-remaining-requests: 9999], [x-ratelimit-remaining-tokens: 199668], [x-ratelimit-reset-requests: 8.64s], [x-ratelimit-reset-tokens: 99ms], [x-request-id: req_e2742c52e7378e5fdfbee75210d57da1], [strict-transport-security: max-age=31536000; includeSubDomains; preload], [CF-Cache-Status: DYNAMIC], [Set-Cookie: __...ne], [X-Content-Type-Options: nosniff], [Set-Cookie: _c...ne], [Server: cloudflare], [CF-RAY: 8de5568209f03b8e-BOS], [alt-svc: h3=":443"; ma=86400]
- body: {
"id": "chatcmpl-AQa87Rzs1zIKEipIhYBVzcs8TXGoG",
"object": "chat.completion",
"created": 1730898775,
"model": "gpt-4o-mini-2024-07-18",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_L3JgC8iYqMNmqYrk8GmE899N",
"type": "function",
"function": {
"name": "getBookingDetails",
"arguments": "{\"firstName\":\"Robert\",\"lastName\":\"Taylor\",\"bookingNumber\":\"105\"}"
}
}
],
"refusal": null
},
"logprobs": null,
"finish_reason": "tool_calls"
}
],
"usage": {
"prompt_tokens": 459,
"completion_tokens": 26,
"total_tokens": 485,
"prompt_tokens_details": {
"cached_tokens": 0,
"audio_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 0,
"audio_tokens": 0,
"accepted_prediction_tokens": 0,
"rejected_prediction_tokens": 0
}
},
"system_fingerprint": "fp_0ba0d124f1"
}
2024-11-06 08:12:56,253 INFO [io.qua.lan.ope.com.OpenAiRestApi$OpenAiClientLogger] (vert.x-eventloop-thread-4) Request:
- method: POST
- url: https://api.openai.com/v1/chat/completions
- headers: [Accept: application/json], [Authorization: Be...1f], [Content-Type: application/json], [User-Agent: langchain4j-openai], [content-length: 4629]
- body: {
"model" : "gpt-4o-mini",
"messages" : [ {
"role" : "system",
"content" : "You are a customer chat support agent of an airline named \"Funnair\".\nRespond in a friendly, helpful, and joyful manner.\nYou are interacting with customers through an online chat system.\nBefore providing information about a booking or cancelling a booking,\nyou MUST ensure you have the following information from the user:\nbooking number, customer first name, and last name.\nCheck the message history for this information before asking the user.\nBefore changing a booking, you MUST ensure it is permitted by the terms.\nIf there is a charge for the change, you MUST ask the user to consent before proceeding.\nUse the provided functions to fetch booking details, change bookings, and cancel bookings.\nToday is 2024-11-06.\n"
}, {
"role" : "user",
"content" : "Hello I'm Robert Taylor and my booking number is 105. I'd like to change my flight date to November 16, 2024.\n\nAnswer using the following information:\n2. Changing Bookings\n- Changes allowed up to 24 hours before flight.\n- Change via online or contact our support.\n- Change fee: $50 for Economy, $30 for Premium Economy, Free for Business Class.\n\n1. Booking Flights\n- Book via our website or mobile app.\n- Full payment required at booking.\n- Ensure accuracy of personal information (Name, ID, etc.) as corrections may incur a $25 fee."
}, {
"role" : "assistant",
"tool_calls" : [ {
"id" : "call_L3JgC8iYqMNmqYrk8GmE899N",
"type" : "function",
"function" : {
"name" : "getBookingDetails",
"arguments" : "{\"firstName\":\"Robert\",\"lastName\":\"Taylor\",\"bookingNumber\":\"105\"}"
}
} ]
}, {
"role" : "tool",
"tool_call_id" : "call_L3JgC8iYqMNmqYrk8GmE899N",
"content" : "{\n \"bookingNumber\" : \"105\",\n \"firstName\" : \"Robert\",\n \"lastName\" : \"Taylor\",\n \"date\" : \"2024-11-14\",\n \"bookingStatus\" : \"CONFIRMED\",\n \"from\" : \"LHR\",\n \"to\" : \"FRA\",\n \"bookingClass\" : \"BUSINESS\"\n}"
} ],
"temperature" : 0.0,
"top_p" : 1.0,
"presence_penalty" : 0.0,
"frequency_penalty" : 0.0,
"tools" : [ {
"type" : "function",
"function" : {
"name" : "getBookingDetails",
"description" : "Retrieves information about an existing booking,\nsuch as the flight date, booking status, departure and arrival airports, and booking class.\n",
"parameters" : {
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName" ]
}
}
}, {
"type" : "function",
"function" : {
"name" : "changeBooking",
"description" : "Modifies an existing booking.\nThis includes making changes to the flight date, and the departure and arrival airports.\n",
"parameters" : {
"type" : "object",
"properties" : {
"newFlightDate" : {
"type" : "object",
"properties" : {
"month" : {
"type" : "integer"
},
"year" : {
"type" : "integer"
},
"day" : {
"type" : "integer"
}
},
"required" : [ ]
},
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"newDepartureAirport" : {
"type" : "string",
"description" : "3-letter code for departure airport"
},
"newArrivalAirport" : {
"type" : "string",
"description" : "3-letter code for arrival airport"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName", "newFlightDate", "newDepartureAirport", "newArrivalAirport" ]
}
}
}, {
"type" : "function",
"function" : {
"name" : "cancelBooking",
"description" : "",
"parameters" : {
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName" ]
}
}
} ]
}
2024-11-06 08:12:58,006 INFO [io.qua.lan.ope.com.OpenAiRestApi$OpenAiClientLogger] (vert.x-eventloop-thread-4) Response:
- status code: 200
- headers: [Date: Wed, 06 Nov 2024 13:12:58 GMT], [Content-Type: application/json], [Content-Length: 1135], [Connection: keep-alive], [access-control-expose-headers: X-Request-ID], [openai-organization: red-hat-jdskfr], [openai-processing-ms: 1674], [openai-version: 2020-10-01], [x-ratelimit-limit-requests: 10000], [x-ratelimit-limit-tokens: 200000], [x-ratelimit-remaining-requests: 9998], [x-ratelimit-remaining-tokens: 199616], [x-ratelimit-reset-requests: 16.56s], [x-ratelimit-reset-tokens: 115ms], [x-request-id: req_3750eb354bf8df94aab4a890156d9605], [strict-transport-security: max-age=31536000; includeSubDomains; preload], [CF-Cache-Status: DYNAMIC], [Set-Cookie: __...ne], [X-Content-Type-Options: nosniff], [Set-Cookie: _c...ne], [Server: cloudflare], [CF-RAY: 8de55687bfb63b8e-BOS], [alt-svc: h3=":443"; ma=86400]
- body: {
"id": "chatcmpl-AQa88rzYQ7ZiKVZljntgHupI5Z3Qw",
"object": "chat.completion",
"created": 1730898776,
"model": "gpt-4o-mini-2024-07-18",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello Robert! 🌟 Thank you for reaching out to Funnair! \n\nI see that your current flight is scheduled for November 14, 2024, and you're looking to change it to November 16, 2024. Since you're in Business Class, I'm happy to inform you that there is no change fee for this modification! \n\nWould you like me to proceed with changing your flight date to November 16, 2024? ✈️",
"refusal": null
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 575,
"completion_tokens": 94,
"total_tokens": 669,
"prompt_tokens_details": {
"cached_tokens": 0,
"audio_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 0,
"audio_tokens": 0,
"accepted_prediction_tokens": 0,
"rejected_prediction_tokens": 0
}
},
"system_fingerprint": "fp_0ba0d124f1"
}
2024-11-06 08:13:10,665 INFO [io.qua.lan.ope.com.OpenAiRestApi$OpenAiClientLogger] (vert.x-eventloop-thread-4) Request:
- method: POST
- url: https://api.openai.com/v1/chat/completions
- headers: [Accept: application/json], [Authorization: Be...1f], [Content-Type: application/json], [User-Agent: langchain4j-openai], [content-length: 5465]
- body: {
"model" : "gpt-4o-mini",
"messages" : [ {
"role" : "system",
"content" : "You are a customer chat support agent of an airline named \"Funnair\".\nRespond in a friendly, helpful, and joyful manner.\nYou are interacting with customers through an online chat system.\nBefore providing information about a booking or cancelling a booking,\nyou MUST ensure you have the following information from the user:\nbooking number, customer first name, and last name.\nCheck the message history for this information before asking the user.\nBefore changing a booking, you MUST ensure it is permitted by the terms.\nIf there is a charge for the change, you MUST ask the user to consent before proceeding.\nUse the provided functions to fetch booking details, change bookings, and cancel bookings.\nToday is 2024-11-06.\n"
}, {
"role" : "user",
"content" : "Hello I'm Robert Taylor and my booking number is 105. I'd like to change my flight date to November 16, 2024.\n\nAnswer using the following information:\n2. Changing Bookings\n- Changes allowed up to 24 hours before flight.\n- Change via online or contact our support.\n- Change fee: $50 for Economy, $30 for Premium Economy, Free for Business Class.\n\n1. Booking Flights\n- Book via our website or mobile app.\n- Full payment required at booking.\n- Ensure accuracy of personal information (Name, ID, etc.) as corrections may incur a $25 fee."
}, {
"role" : "assistant",
"tool_calls" : [ {
"id" : "call_L3JgC8iYqMNmqYrk8GmE899N",
"type" : "function",
"function" : {
"name" : "getBookingDetails",
"arguments" : "{\"firstName\":\"Robert\",\"lastName\":\"Taylor\",\"bookingNumber\":\"105\"}"
}
} ]
}, {
"role" : "tool",
"tool_call_id" : "call_L3JgC8iYqMNmqYrk8GmE899N",
"content" : "{\n \"bookingNumber\" : \"105\",\n \"firstName\" : \"Robert\",\n \"lastName\" : \"Taylor\",\n \"date\" : \"2024-11-14\",\n \"bookingStatus\" : \"CONFIRMED\",\n \"from\" : \"LHR\",\n \"to\" : \"FRA\",\n \"bookingClass\" : \"BUSINESS\"\n}"
}, {
"role" : "assistant",
"content" : "Hello Robert! 🌟 Thank you for reaching out to Funnair! \n\nI see that your current flight is scheduled for November 14, 2024, and you're looking to change it to November 16, 2024. Since you're in Business Class, I'm happy to inform you that there is no change fee for this modification! \n\nWould you like me to proceed with changing your flight date to November 16, 2024? ✈️"
}, {
"role" : "user",
"content" : "Yes please\n\nAnswer using the following information:\nThese Terms of Service govern your experience with Funnair. By booking a flight, you agree to these terms.\n\n1. Booking Flights\n- Book via our website or mobile app.\n- Full payment required at booking.\n- Ensure accuracy of personal information (Name, ID, etc.) as corrections may incur a $25 fee."
} ],
"temperature" : 0.0,
"top_p" : 1.0,
"presence_penalty" : 0.0,
"frequency_penalty" : 0.0,
"tools" : [ {
"type" : "function",
"function" : {
"name" : "getBookingDetails",
"description" : "Retrieves information about an existing booking,\nsuch as the flight date, booking status, departure and arrival airports, and booking class.\n",
"parameters" : {
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName" ]
}
}
}, {
"type" : "function",
"function" : {
"name" : "changeBooking",
"description" : "Modifies an existing booking.\nThis includes making changes to the flight date, and the departure and arrival airports.\n",
"parameters" : {
"type" : "object",
"properties" : {
"newFlightDate" : {
"type" : "object",
"properties" : {
"month" : {
"type" : "integer"
},
"year" : {
"type" : "integer"
},
"day" : {
"type" : "integer"
}
},
"required" : [ ]
},
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"newDepartureAirport" : {
"type" : "string",
"description" : "3-letter code for departure airport"
},
"newArrivalAirport" : {
"type" : "string",
"description" : "3-letter code for arrival airport"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName", "newFlightDate", "newDepartureAirport", "newArrivalAirport" ]
}
}
}, {
"type" : "function",
"function" : {
"name" : "cancelBooking",
"description" : "",
"parameters" : {
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "bookingNumber", "firstName", "lastName" ]
}
}
} ]
}
2024-11-06 08:13:11,690 INFO [io.qua.lan.ope.com.OpenAiRestApi$OpenAiClientLogger] (vert.x-eventloop-thread-4) Response:
- status code: 200
- headers: [Date: Wed, 06 Nov 2024 13:13:11 GMT], [Content-Type: application/json], [Content-Length: 1208], [Connection: keep-alive], [access-control-expose-headers: X-Request-ID], [openai-organization: red-hat-jdskfr], [openai-processing-ms: 849], [openai-version: 2020-10-01], [x-ratelimit-limit-requests: 10000], [x-ratelimit-limit-tokens: 200000], [x-ratelimit-remaining-requests: 9998], [x-ratelimit-remaining-tokens: 199433], [x-ratelimit-reset-requests: 10.787s], [x-ratelimit-reset-tokens: 170ms], [x-request-id: req_ee075b810b6ac3682497a88a3ff83bf9], [strict-transport-security: max-age=31536000; includeSubDomains; preload], [CF-Cache-Status: DYNAMIC], [Set-Cookie: __...ne], [X-Content-Type-Options: nosniff], [Set-Cookie: _c...ne], [Server: cloudflare], [CF-RAY: 8de556e1cb923b8e-BOS], [alt-svc: h3=":443"; ma=86400]
- body: {
"id": "chatcmpl-AQa8MwodWHbOsLrIiRaDlEq3WUTWw",
"object": "chat.completion",
"created": 1730898790,
"model": "gpt-4o-mini-2024-07-18",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_3LqwvxFNObNcT6vvwtIVKAml",
"type": "function",
"function": {
"name": "changeBooking",
"arguments": "{\"newFlightDate\":{\"month\":11,\"year\":2024,\"day\":16},\"firstName\":\"Robert\",\"lastName\":\"Taylor\",\"newDepartureAirport\":\"LHR\",\"newArrivalAirport\":\"FRA\",\"bookingNumber\":\"105\"}"
}
}
],
"refusal": null
},
"logprobs": null,
"finish_reason": "tool_calls"
}
],
"usage": {
"prompt_tokens": 751,
"completion_tokens": 56,
"total_tokens": 807,
"prompt_tokens_details": {
"cached_tokens": 0,
"audio_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 0,
"audio_tokens": 0,
"accepted_prediction_tokens": 0,
"rejected_prediction_tokens": 0
}
},
"system_fingerprint": "fp_0ba0d124f1"
}
2024-11-06 08:13:11,705 ERROR [io.qua.lan.run.too.QuarkusToolExecutor] (executor-thread-3) com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `java.time.LocalDate` from Object value (token `JsonToken.START_OBJECT`)
at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 18] (through reference chain: org.vaadin.marcus.langchain4j.LangChain4jTools$$QuarkusToolArgumentMapper$changeBooking_c316268f90f72bed55786b9940800e8671df667d["newFlightDate"])
2024-11-06 08:13:11,706 ERROR [com.vaa.hil.EndpointInvoker] (executor-thread-3) Endpoint AssistantService method chat execution failure
Exception in AssistantService.java:19
17
18 public String chat(String chatId, String userMessage) {
→ 19 return langChain4JAssistant.chat(chatId, userMessage);
20 }
21 }
: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:118)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at com.vaadin.hilla.EndpointInvoker.invokeVaadinEndpointMethod(EndpointInvoker.java:454)
at com.vaadin.hilla.EndpointInvoker.invoke(EndpointInvoker.java:203)
at com.vaadin.hilla.EndpointController.doServeEndpoint(EndpointController.java:251)
at com.vaadin.hilla.EndpointController.serveEndpoint(EndpointController.java:199)
at com.github.mcollovati.quarkus.hilla.QuarkusEndpointController.serveEndpoint(QuarkusEndpointController.java:79)
at com.github.mcollovati.quarkus.hilla.QuarkusEndpointController$quarkusrestinvoker$serveEndpoint_19ea4aa8e36421f8414cd9ce9157408f2c9d0890.invoke(Unknown Source)
at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:7)
at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.invokeHandler(AbstractResteasyReactiveContext.java:231)
at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
at org.jboss.resteasy.reactive.server.handlers.RestInitialHandler.beginProcessing(RestInitialHandler.java:48)
at io.quarkus.resteasy.reactive.server.servlet.runtime.ResteasyReactiveServlet.service(ResteasyReactiveServlet.java:31)
at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:614)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:63)
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:67)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:133)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:65)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:247)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:111)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:108)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$8$1.call(UndertowDeploymentRecorder.java:643)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:227)
at io.undertow.servlet.handlers.ServletInitialHandler.handleRequest(ServletInitialHandler.java:152)
at io.undertow.server.handlers.CanonicalPathHandler.handleRequest(CanonicalPathHandler.java:49)
at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$1.handleRequest(UndertowDeploymentRecorder.java:126)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:284)
at io.undertow.server.DefaultExchangeHandler.handle(DefaultExchangeHandler.java:18)
at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$4$2.run(UndertowDeploymentRecorder.java:443)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:627)
at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.IllegalArgumentException: params '{"newFlightDate":{"month":11,"year":2024,"day":16},"firstName":"Robert","lastName":"Taylor","newDepartureAirport":"LHR","newArrivalAirport":"FRA","bookingNumber":"105"}' from request do not map onto the parameters needed by 'org.vaadin.marcus.langchain4j.LangChain4jTools_ClientProxy#changeBooking'
at io.quarkiverse.langchain4j.runtime.tool.QuarkusToolExecutor.invalidMethodParams(QuarkusToolExecutor.java:140)
at io.quarkiverse.langchain4j.runtime.tool.QuarkusToolExecutor.prepareArguments(QuarkusToolExecutor.java:95)
at io.quarkiverse.langchain4j.runtime.tool.QuarkusToolExecutor.execute(QuarkusToolExecutor.java:43)
at io.quarkiverse.langchain4j.runtime.tool.QuarkusToolExecutorFactory$1$1.apply(QuarkusToolExecutorFactory.java:37)
at io.quarkiverse.langchain4j.runtime.tool.QuarkusToolExecutorFactory$1$1.apply(QuarkusToolExecutorFactory.java:34)
at io.quarkiverse.langchain4j.runtime.tool.ToolSpanWrapper.wrap(ToolSpanWrapper.java:26)
at io.quarkiverse.langchain4j.runtime.tool.QuarkusToolExecutorFactory$1$2.apply(QuarkusToolExecutorFactory.java:46)
at io.quarkiverse.langchain4j.runtime.tool.QuarkusToolExecutorFactory$1$2.apply(QuarkusToolExecutorFactory.java:43)
at io.quarkiverse.langchain4j.runtime.tool.QuarkusToolExecutorFactory$1.execute(QuarkusToolExecutorFactory.java:52)
at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.doImplement(AiServiceMethodImplementationSupport.java:346)
at io.quarkiverse.langchain4j.runtime.aiservice.AiServiceMethodImplementationSupport.implement(AiServiceMethodImplementationSupport.java:128)
at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1$1.apply(MethodImplementationSupportProducer.java:31)
at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1$1.apply(MethodImplementationSupportProducer.java:28)
at io.quarkiverse.langchain4j.runtime.aiservice.SpanWrapper.wrap(SpanWrapper.java:32)
at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1$2.aphodImplementationSupportProducer.java:40)
at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1$2.apply(MethodImplementationSupportProducer.java:37)
at io.quarkiverse.langchain4j.runtime.aiservice.MethodImplementationSupportProducer$1.implement(MethodImplementationSupportProducer.java:46)
at org.vaadin.marcus.langchain4j.LangChain4jAssistant$$QuarkusImpl.chat(Unknown Source)
at org.vaadin.marcus.langchain4j.LangChain4jAssistant$$QuarkusImpl_ClientProxy.chat(Unknown Source)
at org.vaadin.marcus.client.AssistantService.chat(AssistantService.java:19)
at org.vaadin.marcus.client.AssistantService_ClientProxy.chat(Unknown Source)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
... 54 more"newFlightDate" : {
"type" : "object",
"properties" : {
"month" : {
"type" : "integer"
},
"year" : {
"type" : "integer"
},
"day" : {
"type" : "integer"
}
},
"required" : [ ]
}is exactly what I wanted, thanks.
That's the thing we don't want as LocalDate represents something on its own, we don't want to inspect its content
edeandrea
commented
2 weeks ago FWIW - it works fine using upstream LangChain4j...
024-11-06T08:22:47.043-05:00 DEBUG 88034 --- [.openai.com/...] d.l.service.tool.DefaultToolExecutor : About to execute ToolExecutionRequest { id = "call_hEbb9SJCO087v6O7dllhqILl", name = "changeBooking", arguments = "{"newFlightDate":{"month":11,"year":2024,"day":16},"firstName":"Robert","lastName":"Taylor","newDepartureAirport":"CDG","newArrivalAirport":"SJC","bookingNumber":"105"}" } for memoryId kfpK0APcYeUkLswrS27mc
2024-11-06T08:22:47.048-05:00 DEBUG 88034 --- [.openai.com/...] d.l.service.tool.DefaultToolExecutor : Tool execution result: Success
2024-11-06T08:22:47.051-05:00 DEBUG 88034 --- [.openai.com/...] d.a.openai4j.RequestLoggingInterceptor : Request:
- method: POST
- url: https://api.openai.com/v1/chat/completions
- headers: [Accept: text/event-stream], [Authorization: Bearer sk-su...1f], [User-Agent: langchain4j-openai]
- body: {
"model" : "gpt-4-turbo",
"messages" : [ {
"role" : "system",
"content" : "You are a customer chat support agent of an airline named \"Funnair\".\nRespond in a friendly, helpful, and joyful manner.\nYou are interacting with customers through an online chat system.\nBefore providing information about a booking or cancelling a booking,\nyou MUST ensure you have the following information from the user:\nbooking number, customer first name, and last name.\nCheck the message history for this information before asking the user.\nBefore changing a booking, you MUST ensure it is permitted by the terms.\nIf there is a charge for the change, you MUST ask the user to consent before proceeding.\nUse the provided functions to fetch booking details, change bookings, and cancel bookings.\nToday is 2024-11-06.\n"
}, {
"role" : "user",
"content" : "Hello I'm Robert Taylor and my booking number is 105. I'd like to change my flight date to November 16, 2024.\n\nAnswer using the following information:\n2. Changing Bookings\n- Changes allowed up to 24 hours before flight.\n- Change via online or contact our support.\n- Change fee: $50 for Economy, $30 for Premium Economy, Free for Business Class.\n\n1. Booking Flights\n- Book via our website or mobile app.\n- Full payment required at booking.\n- Ensure accuracy of personal information (Name, ID, etc.) as corrections may incur a $25 fee."
}, {
"role" : "assistant",
"tool_calls" : [ {
"id" : "call_O4DkCII4z5ftvWThCK9STUzd",
"type" : "function",
"function" : {
"name" : "getBookingDetails",
"arguments" : "{\"firstName\":\"Robert\",\"lastName\":\"Taylor\",\"bookingNumber\":\"105\"}"
}
} ]
}, {
"role" : "tool",
"tool_call_id" : "call_O4DkCII4z5ftvWThCK9STUzd",
"content" : "{\n \"bookingNumber\": \"105\",\n \"firstName\": \"Robert\",\n \"lastName\": \"Taylor\",\n \"date\": \"2024-11-14\",\n \"bookingStatus\": \"CONFIRMED\",\n \"from\": \"CDG\",\n \"to\": \"SJC\",\n \"bookingClass\": \"PREMIUM_ECONOMY\"\n}"
}, {
"role" : "assistant",
"content" : "Hello Robert! I see that you'd like to change your flight date to November 16, 2024. Your current flight is scheduled for November 14, 2024, from CDG to SJC in Premium Economy class.\n\nSince you're in Premium Economy, there's a change fee of $30. Would you like to proceed with this change? If so, I'll go ahead and update your booking! 😊"
}, {
"role" : "user",
"content" : "yes please\n\nAnswer using the following information:\nThese Terms of Service govern your experience with Funnair. By booking a flight, you agree to these terms.\n\n1. Booking Flights\n- Book via our website or mobile app.\n- Full payment required at booking.\n- Ensure accuracy of personal information (Name, ID, etc.) as corrections may incur a $25 fee."
}, {
"role" : "assistant",
"tool_calls" : [ {
"id" : "call_hEbb9SJCO087v6O7dllhqILl",
"type" : "function",
"function" : {
"name" : "changeBooking",
"arguments" : "{\"newFlightDate\":{\"month\":11,\"year\":2024,\"day\":16},\"firstName\":\"Robert\",\"lastName\":\"Taylor\",\"newDepartureAirport\":\"CDG\",\"newArrivalAirport\":\"SJC\",\"bookingNumber\":\"105\"}"
}
} ]
}, {
"role" : "tool",
"tool_call_id" : "call_hEbb9SJCO087v6O7dllhqILl",
"content" : "Success"
} ],
"temperature" : 0.0,
"stream" : true,
"stream_options" : {
"include_usage" : true
},
"tools" : [ {
"type" : "function",
"function" : {
"name" : "getBookingDetails",
"description" : "Retrieves information about an existing booking,\nsuch as the flight date, booking status, departure and arrival airports, and booking class.\n",
"strict" : true,
"parameters" : {
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "firstName", "lastName", "bookingNumber" ],
"additionalProperties" : false
}
}
}, {
"type" : "function",
"function" : {
"name" : "changeBooking",
"description" : "Modifies an existing booking.\nThis includes making changes to the flight date, and the departure and arrival airports.\n",
"strict" : true,
"parameters" : {
"type" : "object",
"properties" : {
"newFlightDate" : {
"type" : "object",
"properties" : {
"month" : {
"type" : "integer"
},
"year" : {
"type" : "integer"
},
"day" : {
"type" : "integer"
}
},
"required" : [ "month", "year", "day" ],
"additionalProperties" : false
},
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"newDepartureAirport" : {
"type" : "string",
"description" : "3-letter code for departure airport"
},
"newArrivalAirport" : {
"type" : "string",
"description" : "3-letter code for arrival airport"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "newFlightDate", "firstName", "lastName", "newDepartureAirport", "newArrivalAirport", "bookingNumber" ],
"additionalProperties" : false
}
}
}, {
"type" : "function",
"function" : {
"name" : "cancelBooking",
"description" : "",
"strict" : true,
"parameters" : {
"type" : "object",
"properties" : {
"firstName" : {
"type" : "string"
},
"lastName" : {
"type" : "string"
},
"bookingNumber" : {
"type" : "string"
}
},
"required" : [ "firstName", "lastName", "bookingNumber" ],
"additionalProperties" : false
}
}
} ]
}
Yeah and that is wrong as well :)
Like I said, it works for this case so we can go ahead and include, but I guarantee you it will fail soon for some other JDK type one of us is going to use as a Tool parameter.
edeandrea
commented
2 weeks ago Yeah and that is wrong as well :)
Like I said, it works for this case so we can go ahead and include, but I guarantee you it will fail soon for some other JDK type one of us is going to use as a Tool parameter.
Well right now it works in upstream LangChain4j but fails in Quarkus LangChain4j.
geoand
commented
2 weeks ago We are talking past each other, so let me try and put it another way.
I'll include this now, but your on the hook for providing a proper solution when the next JDK type fails (because of this erroneous way of representing types) 😉
edeandrea
commented
2 weeks ago We are talking past each other, so let me try and put it another way.
Maybe we are, but I'm still confused :)
I'll include this now, but your on the hook for providing a proper solution when the next JDK type fails (because of this erroneous way of representing types) 😉
As it stands, this PR doesn't solve the problem. It successfully describes the parameters, but something is still missing on the receiving side when the tool wants to be invoked. Quarkus LangChain4j doesn't know how to create the object to pass to the tool.
geoand
commented
2 weeks ago As it stands, this PR doesn't solve the problem...
Ah okay, I didn't get that. Wht doesn't it solve it? What issue are you facing?
edeandrea
commented
2 weeks ago This PR as it stands gets Quarkus LangChain4j to describe the parameter, but Quarkus LangChain4j doesn't know how to create the object to pass to the tool, whereas upstream LangChain4j does.
Quarkus barfs on the tool invocation:
Caused by: java.lang.IllegalArgumentException: params '{"newFlightDate":{"month":11,"year":2024,"day":16},"firstName":"Robert","lastName":"Taylor","newDepartureAirport":"SJC","newArrivalAirport":"MUC","bookingNumber":"105"}' from request do not map onto the parameters needed by 'org.vaadin.marcus.langchain4j.LangChain4jTools_ClientProxy#changeBooking'whereas when I run it with pure LangChain4j (using Spring Boot), the tool invocation works fine. LangChain4j knows how to take {"newFlightDate":{"month":11,"year":2024,"day":16} and create a LocalDate from it.
geoand
commented
2 weeks ago I don't know why it would work in upstream LangChain4j as it uses the Jackson to deserialize as well. You'll need to debug it
edeandrea
commented
2 weeks ago I don't know why it would work in upstream LangChain4j as it uses the Jackson to deserialize as well. You'll need to debug it
Well, it does work. I'll see if I can debug it a bit and see what LangChain4j is doing.
geoand
commented
2 weeks ago Yeah, I am not denying it works, just saying that the only way to see why is to debug the working and non working versions
edeandrea
commented
2 weeks ago The other interesting thing is that with upstream I can use tools with streaming responses - something I can't do in Quarkus :/
geoand
commented
2 weeks ago I remember trying tools with streaming response recently and seeing that work (whereas I didn't expect it to)
edeandrea
commented
2 weeks ago I've debugged upstream a bit. All the magic happens in dev.langchain4j.service.tool.DefaultToolExecutor.
Seems for object type arguments LangChain4j delegates to Gson. It converts the argument to Json
and then un-marshals from Json into the object. Gson must know how to do it.
geoand
commented
2 weeks ago Interesting. That means that upstream will soon break as well as it is going to move to Jackson.
One more reason to not use this solution :)
edeandrea
commented
2 weeks ago One more reason to not use this solution :)
Or we can see it as an opportunity to improve what we do :) Let me debug what we're doing a bit.
edeandrea
commented
2 weeks ago 2024-11-06 09:00:33,642 ERROR [io.qua.lan.run.too.QuarkusToolExecutor] (executor-thread-1) com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of typejava.time.LocalDatefrom Object value (tokenJsonToken.START_OBJECT) at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATIONdisabled); line: 1, column: 18] (through reference chain: org.vaadin.marcus.langchain4j.LangChain4jTools$$QuarkusToolArgumentMapper$changeBooking_c316268f90f72bed55786b9940800e8671df667d["newFlightDate"])
edeandrea
commented
2 weeks ago Here's the magic that makes it work in upstream....
private static final Gson GSON = new GsonBuilder()
.setPrettyPrinting()
.registerTypeAdapter(
LocalDate.class,
(JsonSerializer<LocalDate>) (localDate, type, context) ->
new JsonPrimitive(localDate.format(ISO_LOCAL_DATE))
)
.registerTypeAdapter(
LocalDate.class,
(JsonDeserializer<LocalDate>) (json, type, context) -> {
if (json.isJsonObject()) {
JsonObject jsonObject = (JsonObject) json;
int year = jsonObject.get("year").getAsInt();
int month = jsonObject.get("month").getAsInt();
int day = jsonObject.get("day").getAsInt();
return LocalDate.of(year, month, day);
} else {
return LocalDate.parse(json.getAsString(), ISO_LOCAL_DATE);
}
}
)I can just change my tool to be
@Tool("""
Modifies an existing booking.
This includes making changes to the flight date, and the departure and arrival airports.
""")
public void changeBooking(
String bookingNumber,
String firstName,
String lastName,
@P("month of the new flight date") int newFlightDateMonth,
@P("day of the month of the new flight date") int newFlightDateDayOfMonth,
@P("year of the new flight date") int newFlightDateYear,
@P("3-letter code for departure airport") String newDepartureAirport,
@P("3-letter code for arrival airport") String newArrivalAirport
) {
service.changeBooking(bookingNumber, firstName, lastName, LocalDate.of(newFlightDateYear, newFlightDateMonth, newFlightDateDayOfMonth), newDepartureAirport, newArrivalAirport);
}and now everything works :) This is probably a better solution anyways?
geoand
commented
2 weeks ago It's a solution, I wouldn't say it's better :)
geoand
commented
2 weeks ago There is no way around having special handling for some JDK types, because these types actually represent things
Tool parameters - Allow using Jandex for classes that aren't in the initial index
Fixes #1036