eyurtsev / kor

LLM(😽)
https://eyurtsev.github.io/kor/
MIT License
1.57k stars 88 forks source link

Some values are not parsed #259

Closed mann2107 closed 5 months ago

mann2107 commented 5 months ago

I got this response after invoking a chain.

{'text': {'data': {'flight_booking_time_info': [{'departure_from_origin_time': '7 am',
     'return_from_destination_time': '',
     'reach_at_destination_time': '',
     'arrival_at_origin_time': ''}]},
  'raw': 'departure_from_origin_time|return_from_destination_time|reach_at_destination_time|arrival_at_origin_time\n5 pm|7 am|||\n',
  'errors': [],
  'validated_data': {}}}

We can see in "raw" that 5 pm and 7 am are extracted. but in "data" only departure_from_origin_time is outputted.

Here is my Schema:


flight_booking_time_schema = Object(
    id="flight_booking_time_info",
    description="Time-related information about a flight booking request, including departure and arrival preferences.",
    attributes=[
        Text(
            id="departure_from_origin_time",
            description="The preferred time when the traveller wants to depart from the origin location",
            examples=[
                ("I want to depart at 9a", "9a"),
                ("Leaving ideally in the morning", "morning")
            ],
        ),
        Text(
            id="return_from_destination_time",
            description="The preferred time when the traveller wants to return from the destination location",
            examples=[
                ("I prefer returning at 6p", "6p"),
                ("Expected return in the evening", "evening"),                
            ],
        ),
        Text(
            id="reach_at_destination_time",
            description="The preferred time when the traveller wants to reach the destination",
            examples=[
                ("I want to arrive at the Chicago by 5:30 PM", "5:30 PM"),
                ("Prefer to reach by afternoon", "afternoon"),
            ],
        ),
        Text(
            id="arrival_at_origin_time",
            description="The preferred time when the traveller wants to arrive back at the origin location",
            examples=[
                ("I want to come back till 3:00 PM", "3:00 PM"),
                ("Arrival at origin ideally by 7:30 AM", "7:30 AM")
            ],
        ),
    ],
    examples=[
        (
            "I'm looking for a flight departing tomorrow morning and returning at 4:30 PM.",
            [
                {
                    "departure_from_origin_time": "morning", "return_from_destination_time": "4:30 PM", "reach_at_destination_time": "", "arrival_at_origin_time": ""
                }
            ],
        ),
        (
            "I need to arrive at my destination by 6:00 PM tomorrow. Departure should be around 6:30 PM, and return at 9:00 AM next Monday.",
            [
                {
                    "departure_from_origin_time": "6:30 PM", "return_from_destination_time": "9:00 AM", "reach_at_destination_time": "6:00 PM", "arrival_at_origin_time": ""
                }
            ],
        ),
        (
            "I'll need to come back home to Indore on the 25th of February, around 7 am.",
            [
                {
                    "departure_from_origin_time": "", "return_from_destination_time": "", "reach_at_destination_time": "", "arrival_at_origin_time": "7 am"
                }
            ],
        ),
    ],
    many=True,
)

Here is the example I was testing:

Hello there, I hope this message finds you well. I need to book a business trip. I'll need to fly from Bengaluru on 20th Jan 2024 at around 5 pm. After the meeting, I'll be returning from LGA on the 25th of Jan 2024, around 7 am. I would require a hotel and car rental facility also at my destination place. Please confirm the booking as soon as possible. I would prefer American Airlines with Window Seat in the Outbound. Also Book me a Aisle seat in Jet Blue in the return Journey. If You can take care that the fare should not be more than $500 that would be great. Thanks!

mann2107 commented 5 months ago

Also when I changed my model from gpt-3.5-turbo to gpt-4. I am getting the following output

{'text': {'data': {'flight_booking_time_info': [{'departure_from_origin_time': '5 pm',
     'return_from_destination_time': '7 am',
     'reach_at_destination_time': '',
     'arrival_at_origin_time': ''}]},
  'raw': 'departure_from_origin_time|return_from_destination_time|reach_at_destination_time|arrival_at_origin_time\r\n5 pm|7 am||',
  'errors': [],
  'validated_data': {}}}

So, with just an adition of '\r' in the raw output, the mapping of output to entity is working fine.

Comaprison Summary:

Erronous:

{'text': {'data': {'flight_booking_time_info': [{'departure_from_origin_time': '7 am',
'return_from_destination_time': '',
'reach_at_destination_time': '',
'arrival_at_origin_time': ''}]},
'raw': 'departure_from_origin_time|return_from_destination_time|reach_at_destination_time|arrival_at_origin_time\n5 pm|7 am|||\n',
'errors': [],
'validated_data': {}}}

Error Free:

{'text': {'data': {'flight_booking_time_info': [{'departure_from_origin_time': '5 pm',
     'return_from_destination_time': '7 am',
     'reach_at_destination_time': '',
     'arrival_at_origin_time': ''}]},
  'raw': 'departure_from_origin_time|return_from_destination_time|reach_at_destination_time|arrival_at_origin_time\r\n5 pm|7 am||',
  'errors': [],
  'validated_data': {}}}

The only differrence I can see in "raw" is an extra "\r" in the output. Thanks