`with_structured_output` is not working

[aliciayen]

I am trying to use with_structured_output referencing the examples in the function's doc strings:

import os
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_ibm import ChatWatsonx

class AnswerWithJustification(BaseModel):
    answer: str = Field(description="The answer to the query")
    justification: str = Field(description="The justification to the answer")

if __name__ == "__main__":
    llm = ChatWatsonx(
        model_id="mistralai/mixtral-8x7b-instruct-v01",
        url=os.environ["WX_ENDPOINT"],
        project_id=os.environ["WX_PROJECT_ID"],
    )
    structured_llm = llm.with_structured_output(
        AnswerWithJustification,
        include_raw=True
    )

    generated = structured_llm.invoke("What weighs more a pound of bricks or a pound of feathers")

But the output is not expected:

{'raw': AIMessage(content='\n\n```json\n{\n    "type": "function",\n    "function":', response_metadata={'token_usage': {'generated_token_count': 20, 'input_token_count': 546}, 'model_name': 'mistralai/mixtral-8x7b-instruct-v01', 'system_fingerprint': '', 'finish_reason': 'max_tokens'}, id='run-055ce84e-02ea-4283-87a5-e11f8440f407-0', usage_metadata={'input_tokens': 546, 'output_tokens': 20, 'total_tokens': 566}), 'parsed': None, 'parsing_error': None}

Please help advise, thank you in advance!

[aliciayen]

Issue was resolved by redefining max_new_tokens:

from ibm_watson_machine_learning.metanames import GenTextParamsMetaNames as GenParams

llm_params = {
        GenParams.MAX_NEW_TOKENS: 1000,
    }

    llm = ChatWatsonx(
        model_id="mistralai/mixtral-8x7b-instruct-v01",
        url=os.environ["WX_ENDPOINT"],
        project_id=os.environ["WX_PROJECT_ID"],
        params = llm_params