Closed younes-io closed 7 months ago
I'm having the same issue, have you figured it out?
i am having the same issue, did anyone figured it out ? this is the code that is am using : `
from outlines import models from outlines import generate modelOutlines = models.openai("gpt-3.5-turbo") schema = """ { "title": "User", "type": "object", "properties": { "name": {"type": "string"}, "last_name": {"type": "string"}, "id": {"type": "integer"} } } """
generator = generate.json(modelOutlines, schema) result = generator( "Create a user profile with the fields name, last_name and id" ) print(result) `
+1 had the same issue.
+1
+1
NotImplementedError: Cannot use JSON Schema-structure generation with an OpenAI model due to the limitations of the OpenAI API
What are you doing these days to make this work with OpenAI @younes-io ?
+1
+1
Unfortunately OpenAI doesn't provide the interfaces necessary to support structured generation.
Outlines works by updating the token probabilities (softmaxed logits) after each decoder pass such that all illegal tokens have a 0% chance of being selected. OpenAI doesn't support direct logits processing.
To better understand how this grammar guiding library works: can't the logit_bias parameter in the openai completions API be used to restrict and guide the output of the model?
https://platform.openai.com/docs/api-reference/completions/create?lang=curl
@ddvlamin unfortunately not. logit_bias
is constant throughout generation, regardless of what has been generated. You can use this to permanently disable certain tokens while generating, however for structured generation you need to dynamically change which tokens are disabled at each step based on the previously generated tokens.
So is it safe to say that as of now Outlines library does NOT support OpenAI models?
We can do generate.text
, but structured generation (generate.regex
, generate.json
, etc) is not possible with OpenAI's API.
The feature is still in beta, but structured outputs should soon be broadly supported by openai. https://platform.openai.com/docs/guides/structured-outputs https://openai.com/index/introducing-structured-outputs-in-the-api/
Describe the issue as clearly as possible:
I need to generate valid JSON as the output of an email classifier, and I need to be 100% sure that the output will always be valid since it'll be the input of an API call. So, I took the JSON example from the docs and tried it using OpenAI, but it failed.
Steps/code to reproduce the bug:
Expected result:
Error message:
Outlines/Python version information:
Version information
Context for the issue:
I'm building an email classifier for a client, so I need to have a consistent JSON output of the result of classification (a JSON listing the categories and their respective scores); unfortunately, this is not supported due to OpenAI limitations :/