Curly brace ({}) in prompt

[wernerulbts]

How can I execute a prompt, where I tell the model give me back in a specific JSON format?

I know that the { are used for instructions

prompt_template={prompt}

But how I can submit a promot where I give instructions which contains the {.

"John Doe's company, Acme Ltd., is located at 1234 Main Street, Springfield, IL 62704."

Extract all infos in the following format: { "Name": "", "Company": "", "Street": "", "City": "", "State": "", "PostalCode": "" }

Only return your answer in json

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Extract all infos in the following format:\n{\n\"Name\": \"\",\n\"Company\": \"\",\n\"Street\": \"\",\n\"City\": \"\",\n\"State\": \"\",\n\"PostalCode\": \"\"\n}\n\nOnly return your answer in json<|eot_id|><|start_header_id|>user<|end_header_id|>

{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

I get an error with: Prediction failed.

'\n\"Name\"'

[mattt]

Hi @wernerulbts. What you're trying to do can be described as "constrained generation" or "function calling". We don't currently support these features with our official deployments of llama 3, but this model by @hamelsmu demonstrates how you can get with in-context prompting.

As for your specific error, could you please share a URL of a failed prediction and the exact code that you're running?

[wernerulbts]

Hi @mattt , thank you very much I will take a look at it.

Regarding the urls here are two examples, how I tried to add my prompt into the system prompt.

https://replicate.com/p/ea92cnckndrgj0cf73892nqnkr https://replicate.com/p/3w76bfh909rgm0cf7spr23wc5w

When I take a look at Llama 3 https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/ there is something like {{ system_prompt }} for the system prompt.

In your example: <|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>

{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

There is only prompt_template and prompt

My goal is basically have the system_prompt ("You are a helpful assistant") given in a correct way, when my sytem prompt contains "{". Basically, I don't know how to parse the system_prompt in the right way for example using characters like "{" in the system_prompt / prompt_template

[mattt]

@wernerulbts Thanks for sharing that context.

Something you might try to solve the immediate '\n\"Name\"' error you're seeing is to remove the newlines from your system prompt. If the model generates valid JSON, the whitespace can be added with formatting.

Another option, if you're having trouble with {} characters is to encode the information in markup like XML / SGML (<address><name>...</name>...</address>).

Finally, take a look at this blog post I wrote a while back about llama 2 with grammar support, which looks to be similar to what you're trying to do.

[wernerulbts]

@mattt Thank you very much, I also thought about XML format. Your link to the blog post, looks great, I will defiantly test it. A quick work arround, which I found is remove the special characters from the system prompt and add it to the prompt: https://replicate.com/p/nqz3er3ch5rgm0cf7tgbq3wrd4

[mattt]

@wernerulbts So glad to see you got that working! I think what you have there is better than what's described by that blog post, so I'd recommend rolling with that.

Anything else to be done in this issue? Or do you think we're good to mark this as resolved?

[wernerulbts]

@mattt Thank you very much, we can mark it as resolved.