More surported model type

[zhaochenyang20]

In your readme.md, your model and tokenizer are:

model = AutoModelForCausalLM.from_pretrained("databricks/dolly-v2-12b")
tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-12b")

I just want to use a typical T5ForConditionalGeneration model as follows:

"""This module provides a T5 model jsonformer."""

import transformers
from jsonformer import Jsonformer

pretrained_model_name = "t5-small"

model = transformers.T5ForConditionalGeneration.from_pretrained(
    pretrained_model_name
)
tokenizer = transformers.T5Tokenizer.from_pretrained(pretrained_model_name)

json_schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
    }
}

prompt = "Generate a person's information based on the following schema:"
jsonformer = Jsonformer(model, tokenizer, json_schema, prompt)
generated_data = jsonformer()

print(generated_data)

But failed due to the error when generating string:

    111 self.debug("[generate_string] response", response)
    112 split = response.split('"')
--> 113 assert len(split) >= 2
    114 return split[1]

AssertionError:

And I might have to say, could you add more docstring and type hints for your project?

[1rgs]

Unfortunately I think "t5-small" is a bit too small, the model isn't very good at following JSON conventions. it would work in theory, but as above, it seems to want to generate things like {"name": " "}, since we chop/stop at the first generated "

I did revamp the string generation logic to include a start quote in the prompt, removing the requirement that the model generates the start quote. This should make it less brittle, but in order to get sensible outputs, I think going larger on the model is your best bet @zhaochenyang20

[1rgs]

@zhaochenyang20 just published 0.10.0, can you retry with that version? It's not much progress with the model you're using though, but it does not error anymore.

With your example I get {"name": ":"}

[zhaochenyang20]

Thanks!

[zhaochenyang20]

I will see if this works out. And here is a related paper for you:

https://arxiv.org/abs/2304.14293