guidance-ai / guidance

A guidance language for controlling large language models.
MIT License
18.8k stars 1.04k forks source link

How to dynamically generate using guidance? #908

Closed LuoKaiGSW closed 3 months ago

LuoKaiGSW commented 3 months ago

I currently need the model to extract a JSON segment based on the provided json_schema and query, but I find that the performance of guidance.json does not meet expectations. It might be because this model has been fine-tuned on specifically formatted data, such as outputting different fields in JSON will cause line breaks, which is inconsistent with the output of guidance. Therefore, I want to generate the content of all fields in the JSON using functions such as gen and select. My approach is to write a function that outputs an f-string including gen and select functions when given a json_schema (the result is represented by ss). However, when I execute lm + ss, the model does not execute the internal functions; if I copy the content of the string, I can get the desired result.

ss = f"""
      "tool_args": {{
        "eventId": "{gen(name="eventId",stop ='"')}",
        "eventName": "{gen(name="eventName",stop ='"')}",
        "active": {select(options=["true","false"], name="active")},
        "participants": [{gen(name="participants",stop ="]")}],
        "sportType": {select(options=['足球', '篮球', '田径', '游泳', '其他'], name="sportType")},
        "location": {{{gen(name="location",stop ="}")}}},
     }}"""

lm += ss # Does not execute the functions within

lm += f"""
      "tool_args": {{
        "eventId": "{gen(name="eventId",stop ='"')}",
        "eventName": "{gen(name="eventName",stop ='"')}",
        "active": {select(options=["true","false"], name="active")},
        "participants": [{gen(name="participants",stop ="]")}],
        "sportType": {select(options=['足球', '篮球', '田径', '游泳', '其他'], name="sportType")},
        "location": {{{gen(name="location",stop ="}")}}},
     }}"""  # This can produce the desired result
LuoKaiGSW commented 3 months ago

This issue has been resolved.