Closed joshpopelka20 closed 5 days ago
cc @interstellarninja
He's using Meta's model NousResearch/Meta-Llama-3-8B-Instruct, not fine-tuned with our format 😅
I'm using
NousResearch/Meta-Llama-3-8B-Instruct
with JSON output. I added the system message template as specified in the docs:{"role": "system", "content": "You are a helpful assistant that only answers in JSON. Here's the json schema you must adhere to:\n<schema>\n{{" + pydantic_schema + "}}\n<schema>\n"},
This is my schema:
class Classification(BaseModel): text: str classification: str class Config: json_schema_extra = { "additionalProperties": False } # serialize pydantic model into json schema pydantic_schema = Classification.schema_json()
My issue is that the output doesn't return just the JSON output. It returns an additional chat message:
'{"text": "3 has been determined to be evolving", "classification": "REPORTABLE"}\n\n{"text": "3 has been determined to be evolving", "classification": "REPORTABLE"}\nThe sentence in the context that most closely matches the given sentence is: "3 has been determined to be evolving" with a classification of "REPORTABLE".\nJSON Response: {"text": "3 has been determined to be evolving", "classification": "REPORTABLE"}\nNote: The JSON response provided is already in the required format. No further action is needed.\nJSON Response: {"text": "3 has been determined to be evolving", "classification": "REPORTABLE"}\nThe sentence in the context that most closely matches the given sentence is: "3 has been determined to be evolving" with a classification of "REPORTABLE".\nJSON Response: {"text": "3 has been determined to be evolving", "classification": "REPORTABLE"}\nNote: The JSON response provided is already in the required format. No further action is needed.\nJSON Response: {"text": "3 has been determined to be evolving", "classification": "REPORTABLE"}\nThe sentence in the context that most closely matches'
How do I get it to only output a message like this:
{"text": "3 has been determined to be evolving", "classification": "REPORTABLE"}
Please use NousResearch/Hermes-2-Pro-Llama-3-8B
I'm using
NousResearch/Meta-Llama-3-8B-Instruct
with JSON output. I added the system message template as specified in the docs:This is my schema:
My issue is that the output doesn't return just the JSON output. It returns an additional chat message:
How do I get it to only output a message like this: