Not sure if I am using Guards correctly.

[oliverbj]

Consider the below instruction + rail spec:

query = "new booking from hong kong to london with 2 pallets in a 40 HC"

rail_spec = """
<rail version="0.1">

<output strict="true">
    <object name="shipment" format="length: 4">
        <string name="origin_city" description="The city name of the origin of the shipment" />
        <string name="orirgin_country" description="The country name of the origin of the shipment" />
        <string name="destination_city" description="The city name of the destination of the shipment" />
        <string name="destination_country" description="The country name of the destination of the shipment (can be taken from the destination_city)" />
    </object>
</output>

<prompt>
Generate a valid JSON object for a shipment given a users query:

{{query}}

@complete_json_suffix
</prompt>
</rail>
"""

This will produce the following JSON:

{'origin_city': 'Hong Kong', 'origin_country': 'China', 'destination_city': 'London', 'destination_country': 'United Kingdom', 'pallets': 2, 'container_type': '40 HC'}

As you can see, pallets and container_type was added to the output, even though it is not defined in my output.

[ShreyaR]

Hey @oliverbj,

Thanks for flagging this issue, I think you just found guardrails' the first bug!

You are using the Guard module correctly, but the output generated by the LLM isn't respecting the provided schema. This should ordinarily be taken care of by Guardrails, and there are two planned fixes on the roadmap this week that should address this issue:

1. in-context examples to force the model to follow the schema (issue #14)
2. bug fix in output validation for object type (issue #15)

In the meantime, I got the example working by removing the outermost shipment element from the rail spec:

rail_spec = """
<rail version="0.1">

<output strict="true">
    <string name="origin_city" description="The city name of the origin of the shipment" />
    <string name="orirgin_country" description="The country name of the origin of the shipment" />
    <string name="destination_city" description="The city name of the destination of the shipment" />
    <string name="destination_country" description="The country name of the destination of the shipment (can be taken from the destination_city)" />
</output>

<prompt>
Generate a valid JSON object for a shipment given a users query:

{{query}}

@complete_json_suffix
</prompt>
</rail>
"""

Hope this helps! I should have the issues I linked resolved soon. Let me know if you run into any other problems.