create_extraction_chain does not work with other LLMs? Replicate models fails to load

[codenamics]

System Info

LangChain 0.0.281

Who can help?

No response

Information

• [ ] The official example notebooks/scripts
• [X] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [X] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

Use this lines of code

schema = {
    "properties": {
        "visit": {"type": "string"},
        "date": {"type": "string"},
        "gender": {"type": "string"},
        "age": {"type": "integer"},
    }

}

inp = """This 23-year-old white female presents with complaint of allergies. 
She used to have allergies when she lived in Seattle but she thinks they are worse here. 
In the past, she has tried Claritin, and Zyrtec. Both worked for short time but then seemed to lose effectiveness.  """

llm = Replicate(
    model="a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5",
    input={"temperature": 0.75, "max_length": 500, "top_p": 1},
)
chain = create_extraction_chain(schema, llm)
chain.run(inp)

---

OutputParserException                     Traceback (most recent call last)
[<ipython-input-9-5e77f11609b2>](https://localhost:8080/#) in <cell line: 72>()
     70 )
     71 chain = create_extraction_chain(schema, llm)
---> 72 chain.run(inp)["data"]

8 frames
[/usr/local/lib/python3.10/dist-packages/langchain/output_parsers/openai_functions.py](https://localhost:8080/#) in parse_result(self, result)
     21         generation = result[0]
     22         if not isinstance(generation, ChatGeneration):
---> 23             raise OutputParserException(
     24                 "This output parser can only be used with a chat generation."
     25             )

OutputParserException: This output parser can only be used with a chat generation.

Expected behavior

Structrued JSON based on schema

[dosubot[bot]]

🤖

Hello,

Thank you for reaching out and providing detailed information about the issue you're facing.

The create_extraction_chain function is designed to work with specific language learning models (LLMs) and it seems like the Replicate model you're trying to use might not be fully compatible with it. The function might be using specific methods or properties that are only available in certain LLMs, and not in the Replicate model.

As for the Replicate model failing to load, it could be due to a missing or incorrect REPLICATE_API_TOKEN. The Replicate model requires the REPLICATE_API_TOKEN environment variable to be set with your API token. If the token is missing or incorrect, the model will fail to load and result in an OutputParserException.

Another possible reason for the Replicate model failing to load could be the absence of the replicate Python package. The Replicate model requires the replicate Python package to be installed. If it's not installed, an ImportError will be raised.

Please ensure that the REPLICATE_API_TOKEN is correctly set and the replicate Python package is installed. You can install the replicate package using pip:

pip install replicate

If you've already done these steps and are still facing the issue, it would be helpful if you could provide more information about your environment, such as the Python version you're using and the list of installed packages (pip freeze).

I hope this helps! Let me know if you have any other questions or if there's anything else I can do to assist you.

Best, Dosu

Sources

• libs/langchain/langchain/llms/replicate.py
• libs/langchain/langchain/chains/llm.py
• libs/langchain/langchain/chains/openai_functions/extraction.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[braun-viathan]

Same for me. I want to use azure openai models with extraction.

Help is appreciated, thank you.

CODE:

azure_model = AzureChatOpenAI(
    openai_api_type="azure",
    engine="gpt35-playground",
    deployment_name="gpt35-playground",
    # place model settings from the playground here
    temperature=0,
    max_tokens=150,
    top_p=0,
    frequency_penalty=0,
    presence_penalty=0,
    stop=None,
    model_version="0301"
)

from langchain.chat_models import ChatOpenAI
from langchain.chains import create_extraction_chain

# Schema
schema = {
    "properties": {
        "name": {"type": "string"},
        "height": {"type": "integer"},
        "hair_color": {"type": "string"},
    },
    "required": ["name", "height"],
}

inp = """Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde."""

chain = create_extraction_chain(schema, llm=azure_model)
chain.run(inp)

ERROR:

---------------------------------------------------------------------------
InvalidRequestError                       Traceback (most recent call last)
[/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb) Cell 4 line 2
     [17](vscode-notebook-cell:/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb#X20sZmlsZQ%3D%3D?line=16) # Run chain
     [19](vscode-notebook-cell:/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb#X20sZmlsZQ%3D%3D?line=18) chain = create_extraction_chain(schema, llm=model)
---> [20](vscode-notebook-cell:/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb#X20sZmlsZQ%3D%3D?line=19) chain.run(inp)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:501](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:501), in Chain.run(self, callbacks, tags, metadata, *args, **kwargs)
    499     if len(args) != 1:
    500         raise ValueError("`run` supports only one positional argument.")
--> 501     return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
    502         _output_key
    503     ]
    505 if kwargs and not args:
    506     return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
    507         _output_key
    508     ]

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:306](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:306), in Chain.__call__(self, inputs, return_only_outputs, callbacks, tags, metadata, run_name, include_run_info)
    304 except BaseException as e:
    305     run_manager.on_chain_error(e)
--> 306     raise e
    307 run_manager.on_chain_end(outputs)
    308 final_outputs: Dict[str, Any] = self.prep_outputs(
    309     inputs, outputs, return_only_outputs
    310 )

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:300](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:300), in Chain.__call__(self, inputs, return_only_outputs, callbacks, tags, metadata, run_name, include_run_info)
    293 run_manager = callback_manager.on_chain_start(
    294     dumpd(self),
    295     inputs,
    296     name=run_name,
    297 )
    298 try:
    299     outputs = (
--> 300         self._call(inputs, run_manager=run_manager)
    301         if new_arg_supported
    302         else self._call(inputs)
    303     )
    304 except BaseException as e:
    305     run_manager.on_chain_error(e)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/llm.py:93](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/llm.py:93), in LLMChain._call(self, inputs, run_manager)
     88 def _call(
     89     self,
     90     inputs: Dict[str, Any],
     91     run_manager: Optional[CallbackManagerForChainRun] = None,
     92 ) -> Dict[str, str]:
---> 93     response = self.generate([inputs], run_manager=run_manager)
     94     return self.create_outputs(response)[0]

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/llm.py:103](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/llm.py:103), in LLMChain.generate(self, input_list, run_manager)
    101 """Generate LLM result from inputs."""
    102 prompts, stop = self.prep_prompts(input_list, run_manager=run_manager)
--> 103 return self.llm.generate_prompt(
    104     prompts,
    105     stop,
    106     callbacks=run_manager.get_child() if run_manager else None,
    107     **self.llm_kwargs,
    108 )

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:469](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:469), in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, **kwargs)
    461 def generate_prompt(
    462     self,
    463     prompts: List[PromptValue],
   (...)
    466     **kwargs: Any,
    467 ) -> LLMResult:
    468     prompt_messages = [p.to_messages() for p in prompts]
--> 469     return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:359](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:359), in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, **kwargs)
    357         if run_managers:
    358             run_managers[i].on_llm_error(e)
--> 359         raise e
    360 flattened_outputs = [
    361     LLMResult(generations=[res.generations], llm_output=res.llm_output)
    362     for res in results
    363 ]
    364 llm_output = self._combine_llm_outputs([res.llm_output for res in results])

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:349](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:349), in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, **kwargs)
    346 for i, m in enumerate(messages):
    347     try:
    348         results.append(
--> 349             self._generate_with_cache(
    350                 m,
    351                 stop=stop,
    352                 run_manager=run_managers[i] if run_managers else None,
    353                 **kwargs,
    354             )
    355         )
    356     except BaseException as e:
    357         if run_managers:

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:501](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:501), in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, **kwargs)
    497     raise ValueError(
    498         "Asked to cache, but no cache found at `langchain.cache`."
    499     )
    500 if new_arg_supported:
--> 501     return self._generate(
    502         messages, stop=stop, run_manager=run_manager, **kwargs
    503     )
    504 else:
    505     return self._generate(messages, stop=stop, **kwargs)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:345](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:345), in ChatOpenAI._generate(self, messages, stop, run_manager, stream, **kwargs)
    343 message_dicts, params = self._create_message_dicts(messages, stop)
    344 params = {**params, **kwargs}
--> 345 response = self.completion_with_retry(
    346     messages=message_dicts, run_manager=run_manager, **params
    347 )
    348 return self._create_chat_result(response)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:284](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:284), in ChatOpenAI.completion_with_retry(self, run_manager, **kwargs)
    280 @retry_decorator
    281 def _completion_with_retry(**kwargs: Any) -> Any:
    282     return self.client.create(**kwargs)
--> 284 return _completion_with_retry(**kwargs)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:289](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:289), in BaseRetrying.wraps.<locals>.wrapped_f(*args, **kw)
    287 @functools.wraps(f)
    288 def wrapped_f(*args: t.Any, **kw: t.Any) -> t.Any:
--> 289     return self(f, *args, **kw)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:379](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:379), in Retrying.__call__(self, fn, *args, **kwargs)
    377 retry_state = RetryCallState(retry_object=self, fn=fn, args=args, kwargs=kwargs)
    378 while True:
--> 379     do = self.iter(retry_state=retry_state)
    380     if isinstance(do, DoAttempt):
    381         try:

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:314](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:314), in BaseRetrying.iter(self, retry_state)
    312 is_explicit_retry = fut.failed and isinstance(fut.exception(), TryAgain)
    313 if not (is_explicit_retry or self.retry(retry_state)):
--> 314     return fut.result()
    316 if self.after is not None:
    317     self.after(retry_state)

File [/usr/lib/python3.10/concurrent/futures/_base.py:451](https://file+.vscode-resource.vscode-cdn.net/usr/lib/python3.10/concurrent/futures/_base.py:451), in Future.result(self, timeout)
    449     raise CancelledError()
    450 elif self._state == FINISHED:
--> 451     return self.__get_result()
    453 self._condition.wait(timeout)
    455 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:

File [/usr/lib/python3.10/concurrent/futures/_base.py:403](https://file+.vscode-resource.vscode-cdn.net/usr/lib/python3.10/concurrent/futures/_base.py:403), in Future.__get_result(self)
    401 if self._exception:
    402     try:
--> 403         raise self._exception
    404     finally:
    405         # Break a reference cycle with the exception in self._exception
    406         self = None

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:382](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:382), in Retrying.__call__(self, fn, *args, **kwargs)
    380 if isinstance(do, DoAttempt):
    381     try:
--> 382         result = fn(*args, **kwargs)
    383     except BaseException:  # noqa: B902
    384         retry_state.set_exception(sys.exc_info())  # type: ignore[arg-type]

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:282](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:282), in ChatOpenAI.completion_with_retry.<locals>._completion_with_retry(**kwargs)
    280 @retry_decorator
    281 def _completion_with_retry(**kwargs: Any) -> Any:
--> 282     return self.client.create(**kwargs)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_resources/chat_completion.py:25](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_resources/chat_completion.py:25), in ChatCompletion.create(cls, *args, **kwargs)
     23 while True:
     24     try:
---> 25         return super().create(*args, **kwargs)
     26     except TryAgain as e:
     27         if timeout is not None and time.time() > start + timeout:

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py:155](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py:155), in EngineAPIResource.create(cls, api_key, api_base, api_type, request_id, api_version, organization, **params)
    129 @classmethod
    130 def create(
    131     cls,
   (...)
    138     **params,
    139 ):
    140     (
    141         deployment_id,
    142         engine,
   (...)
    152         api_key, api_base, api_type, api_version, organization, **params
    153     )
--> 155     response, _, api_key = requestor.request(
    156         "post",
    157         url,
    158         params=params,
    159         headers=headers,
    160         stream=stream,
    161         request_id=request_id,
    162         request_timeout=request_timeout,
    163     )
    165     if stream:
    166         # must be an iterator
    167         assert not isinstance(response, OpenAIResponse)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:299](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:299), in APIRequestor.request(self, method, url, params, headers, files, stream, request_id, request_timeout)
    278 def request(
    279     self,
    280     method,
   (...)
    287     request_timeout: Optional[Union[float, Tuple[float, float]]] = None,
    288 ) -> Tuple[Union[OpenAIResponse, Iterator[OpenAIResponse]], bool, str]:
    289     result = self.request_raw(
    290         method.lower(),
    291         url,
   (...)
    297         request_timeout=request_timeout,
    298     )
--> 299     resp, got_stream = self._interpret_response(result, stream)
    300     return resp, got_stream, self.api_key

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:710](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:710), in APIRequestor._interpret_response(self, result, stream)
    702     return (
    703         self._interpret_response_line(
    704             line, result.status_code, result.headers, stream=True
    705         )
    706         for line in parse_stream(result.iter_lines())
    707     ), True
    708 else:
    709     return (
--> 710         self._interpret_response_line(
    711             result.content.decode("utf-8"),
    712             result.status_code,
    713             result.headers,
    714             stream=False,
    715         ),
    716         False,
    717     )

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:775](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:775), in APIRequestor._interpret_response_line(self, rbody, rcode, rheaders, stream)
    773 stream_error = stream and "error" in resp.data
    774 if stream_error or not 200 <= rcode < 300:
--> 775     raise self.handle_error_response(
    776         rbody, rcode, resp.data, rheaders, stream_error=stream_error
    777     )
    778 return resp

InvalidRequestError: Unrecognized request argument supplied: functions

[kaushalpowar]

Has anyone solved this issue?

[arieltoledo]

Year 2023 Same Question.

Code

repo_id="mistralai/Mistral-7B-Instruct-v0.1"

    llm = HuggingFaceHub(repo_id=repo_id)

    class Properties(BaseModel):
        person_name: str
        person_height: int
        person_hair_color: str
        dog_breed: Optional[str]
        dog_name: Optional[str]

    schema = {
    "properties": {
        "person_name": {"type": "string"},
        "person_height": {"type": "integer"},
        "person_hair_color": {"type": "string"},
        "dog_name": {"type": "string"},
        "dog_breed": {"type": "string"},
        "dog_extra_info": {"type": "string"},
    },
}
chain = create_extraction_chain_pydantic(pydantic_schema=Properties, llm=llm)

inp = """Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde."""

chain.run(inp)

ISSUE

raise OutputParserException( langchain_core.exceptions.OutputParserException: This output parser can only be used with a chat generation.

[georgiiic]

Any chance one of you managed to find a solution?

[dsmatthew]

Any chance one of you managed to find a solution?

I tried to use LangChain + Azure OpenAI. My workaround is the library Kor. Kor comes with a few bugs, but maybe you can give it a try.

[Yanni8]

Year 2023 Same Question.

Code

repo_id="mistralai/Mistral-7B-Instruct-v0.1"

    llm = HuggingFaceHub(repo_id=repo_id)

    class Properties(BaseModel):
        person_name: str
        person_height: int
        person_hair_color: str
        dog_breed: Optional[str]
        dog_name: Optional[str]

    schema = {
    "properties": {
        "person_name": {"type": "string"},
        "person_height": {"type": "integer"},
        "person_hair_color": {"type": "string"},
        "dog_name": {"type": "string"},
        "dog_breed": {"type": "string"},
        "dog_extra_info": {"type": "string"},
    },
}
chain = create_extraction_chain_pydantic(pydantic_schema=Properties, llm=llm)

inp = """Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde."""

chain.run(inp)

ISSUE

raise OutputParserException( langchain_core.exceptions.OutputParserException: This output parser can only be used with a chat generation.

@arieltoledo

The documentation and error message are definitely not good, but the issue is quite simple (at least I think that this might be the issue). What you are trying to do won't work since you are creating an Agent that uses OpenAI Functions (https://platform.openai.com/docs/guides/function-calling) but with a Chat Model that doesn't support them.

That's also what the error message is trying to tell you. This output parser can only be used with a chat generation → This Output Parser does only work with OpenAI Models.

The solution to the problem would probably be to use the 2. Approach by using a Parser. You can find the documentation here: https://python.langchain.com/docs/use_cases/extraction#option-2-parsing

I haven't tried it out yet, but I think that this might help you.

[georgiiic]

It was indeed the problem. I managed to make it work wit the Kor library, which is just another way to create output parsors.

Thanks a lot for your response & help.

Really appreciate it !

[Yuvaraj2519]

Same for me. I want to use azure openai models with extraction.

Help is appreciated, thank you.

CODE:

azure_model = AzureChatOpenAI(
    openai_api_type="azure",
    engine="gpt35-playground",
    deployment_name="gpt35-playground",
    # place model settings from the playground here
    temperature=0,
    max_tokens=150,
    top_p=0,
    frequency_penalty=0,
    presence_penalty=0,
    stop=None,
    model_version="0301"
)

from langchain.chat_models import ChatOpenAI
from langchain.chains import create_extraction_chain

# Schema
schema = {
    "properties": {
        "name": {"type": "string"},
        "height": {"type": "integer"},
        "hair_color": {"type": "string"},
    },
    "required": ["name", "height"],
}

inp = """Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde."""

chain = create_extraction_chain(schema, llm=azure_model)
chain.run(inp)

ERROR:

---------------------------------------------------------------------------
InvalidRequestError                       Traceback (most recent call last)
[/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb) Cell 4 line 2
     [17](vscode-notebook-cell:/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb#X20sZmlsZQ%3D%3D?line=16) # Run chain
     [19](vscode-notebook-cell:/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb#X20sZmlsZQ%3D%3D?line=18) chain = create_extraction_chain(schema, llm=model)
---> [20](vscode-notebook-cell:/home/philipp/dev/ai-vertragsdaten-production/tests/output_parser.ipynb#X20sZmlsZQ%3D%3D?line=19) chain.run(inp)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:501](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:501), in Chain.run(self, callbacks, tags, metadata, *args, **kwargs)
    499     if len(args) != 1:
    500         raise ValueError("`run` supports only one positional argument.")
--> 501     return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
    502         _output_key
    503     ]
    505 if kwargs and not args:
    506     return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
    507         _output_key
    508     ]

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:306](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:306), in Chain.__call__(self, inputs, return_only_outputs, callbacks, tags, metadata, run_name, include_run_info)
    304 except BaseException as e:
    305     run_manager.on_chain_error(e)
--> 306     raise e
    307 run_manager.on_chain_end(outputs)
    308 final_outputs: Dict[str, Any] = self.prep_outputs(
    309     inputs, outputs, return_only_outputs
    310 )

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:300](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/base.py:300), in Chain.__call__(self, inputs, return_only_outputs, callbacks, tags, metadata, run_name, include_run_info)
    293 run_manager = callback_manager.on_chain_start(
    294     dumpd(self),
    295     inputs,
    296     name=run_name,
    297 )
    298 try:
    299     outputs = (
--> 300         self._call(inputs, run_manager=run_manager)
    301         if new_arg_supported
    302         else self._call(inputs)
    303     )
    304 except BaseException as e:
    305     run_manager.on_chain_error(e)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/llm.py:93](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/llm.py:93), in LLMChain._call(self, inputs, run_manager)
     88 def _call(
     89     self,
     90     inputs: Dict[str, Any],
     91     run_manager: Optional[CallbackManagerForChainRun] = None,
     92 ) -> Dict[str, str]:
---> 93     response = self.generate([inputs], run_manager=run_manager)
     94     return self.create_outputs(response)[0]

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/llm.py:103](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chains/llm.py:103), in LLMChain.generate(self, input_list, run_manager)
    101 """Generate LLM result from inputs."""
    102 prompts, stop = self.prep_prompts(input_list, run_manager=run_manager)
--> 103 return self.llm.generate_prompt(
    104     prompts,
    105     stop,
    106     callbacks=run_manager.get_child() if run_manager else None,
    107     **self.llm_kwargs,
    108 )

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:469](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:469), in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, **kwargs)
    461 def generate_prompt(
    462     self,
    463     prompts: List[PromptValue],
   (...)
    466     **kwargs: Any,
    467 ) -> LLMResult:
    468     prompt_messages = [p.to_messages() for p in prompts]
--> 469     return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:359](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:359), in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, **kwargs)
    357         if run_managers:
    358             run_managers[i].on_llm_error(e)
--> 359         raise e
    360 flattened_outputs = [
    361     LLMResult(generations=[res.generations], llm_output=res.llm_output)
    362     for res in results
    363 ]
    364 llm_output = self._combine_llm_outputs([res.llm_output for res in results])

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:349](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:349), in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, **kwargs)
    346 for i, m in enumerate(messages):
    347     try:
    348         results.append(
--> 349             self._generate_with_cache(
    350                 m,
    351                 stop=stop,
    352                 run_manager=run_managers[i] if run_managers else None,
    353                 **kwargs,
    354             )
    355         )
    356     except BaseException as e:
    357         if run_managers:

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:501](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/base.py:501), in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, **kwargs)
    497     raise ValueError(
    498         "Asked to cache, but no cache found at `langchain.cache`."
    499     )
    500 if new_arg_supported:
--> 501     return self._generate(
    502         messages, stop=stop, run_manager=run_manager, **kwargs
    503     )
    504 else:
    505     return self._generate(messages, stop=stop, **kwargs)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:345](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:345), in ChatOpenAI._generate(self, messages, stop, run_manager, stream, **kwargs)
    343 message_dicts, params = self._create_message_dicts(messages, stop)
    344 params = {**params, **kwargs}
--> 345 response = self.completion_with_retry(
    346     messages=message_dicts, run_manager=run_manager, **params
    347 )
    348 return self._create_chat_result(response)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:284](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:284), in ChatOpenAI.completion_with_retry(self, run_manager, **kwargs)
    280 @retry_decorator
    281 def _completion_with_retry(**kwargs: Any) -> Any:
    282     return self.client.create(**kwargs)
--> 284 return _completion_with_retry(**kwargs)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:289](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:289), in BaseRetrying.wraps.<locals>.wrapped_f(*args, **kw)
    287 @functools.wraps(f)
    288 def wrapped_f(*args: t.Any, **kw: t.Any) -> t.Any:
--> 289     return self(f, *args, **kw)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:379](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:379), in Retrying.__call__(self, fn, *args, **kwargs)
    377 retry_state = RetryCallState(retry_object=self, fn=fn, args=args, kwargs=kwargs)
    378 while True:
--> 379     do = self.iter(retry_state=retry_state)
    380     if isinstance(do, DoAttempt):
    381         try:

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:314](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:314), in BaseRetrying.iter(self, retry_state)
    312 is_explicit_retry = fut.failed and isinstance(fut.exception(), TryAgain)
    313 if not (is_explicit_retry or self.retry(retry_state)):
--> 314     return fut.result()
    316 if self.after is not None:
    317     self.after(retry_state)

File [/usr/lib/python3.10/concurrent/futures/_base.py:451](https://file+.vscode-resource.vscode-cdn.net/usr/lib/python3.10/concurrent/futures/_base.py:451), in Future.result(self, timeout)
    449     raise CancelledError()
    450 elif self._state == FINISHED:
--> 451     return self.__get_result()
    453 self._condition.wait(timeout)
    455 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:

File [/usr/lib/python3.10/concurrent/futures/_base.py:403](https://file+.vscode-resource.vscode-cdn.net/usr/lib/python3.10/concurrent/futures/_base.py:403), in Future.__get_result(self)
    401 if self._exception:
    402     try:
--> 403         raise self._exception
    404     finally:
    405         # Break a reference cycle with the exception in self._exception
    406         self = None

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:382](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/tenacity/__init__.py:382), in Retrying.__call__(self, fn, *args, **kwargs)
    380 if isinstance(do, DoAttempt):
    381     try:
--> 382         result = fn(*args, **kwargs)
    383     except BaseException:  # noqa: B902
    384         retry_state.set_exception(sys.exc_info())  # type: ignore[arg-type]

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:282](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/langchain/chat_models/openai.py:282), in ChatOpenAI.completion_with_retry.<locals>._completion_with_retry(**kwargs)
    280 @retry_decorator
    281 def _completion_with_retry(**kwargs: Any) -> Any:
--> 282     return self.client.create(**kwargs)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_resources/chat_completion.py:25](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_resources/chat_completion.py:25), in ChatCompletion.create(cls, *args, **kwargs)
     23 while True:
     24     try:
---> 25         return super().create(*args, **kwargs)
     26     except TryAgain as e:
     27         if timeout is not None and time.time() > start + timeout:

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py:155](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py:155), in EngineAPIResource.create(cls, api_key, api_base, api_type, request_id, api_version, organization, **params)
    129 @classmethod
    130 def create(
    131     cls,
   (...)
    138     **params,
    139 ):
    140     (
    141         deployment_id,
    142         engine,
   (...)
    152         api_key, api_base, api_type, api_version, organization, **params
    153     )
--> 155     response, _, api_key = requestor.request(
    156         "post",
    157         url,
    158         params=params,
    159         headers=headers,
    160         stream=stream,
    161         request_id=request_id,
    162         request_timeout=request_timeout,
    163     )
    165     if stream:
    166         # must be an iterator
    167         assert not isinstance(response, OpenAIResponse)

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:299](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:299), in APIRequestor.request(self, method, url, params, headers, files, stream, request_id, request_timeout)
    278 def request(
    279     self,
    280     method,
   (...)
    287     request_timeout: Optional[Union[float, Tuple[float, float]]] = None,
    288 ) -> Tuple[Union[OpenAIResponse, Iterator[OpenAIResponse]], bool, str]:
    289     result = self.request_raw(
    290         method.lower(),
    291         url,
   (...)
    297         request_timeout=request_timeout,
    298     )
--> 299     resp, got_stream = self._interpret_response(result, stream)
    300     return resp, got_stream, self.api_key

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:710](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:710), in APIRequestor._interpret_response(self, result, stream)
    702     return (
    703         self._interpret_response_line(
    704             line, result.status_code, result.headers, stream=True
    705         )
    706         for line in parse_stream(result.iter_lines())
    707     ), True
    708 else:
    709     return (
--> 710         self._interpret_response_line(
    711             result.content.decode("utf-8"),
    712             result.status_code,
    713             result.headers,
    714             stream=False,
    715         ),
    716         False,
    717     )

File [~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:775](https://file+.vscode-resource.vscode-cdn.net/home/philipp/dev/ai-vertragsdaten-production/tests/~/dev/ai-vertragsdaten-production/.venv/lib/python3.10/site-packages/openai/api_requestor.py:775), in APIRequestor._interpret_response_line(self, rbody, rcode, rheaders, stream)
    773 stream_error = stream and "error" in resp.data
    774 if stream_error or not 200 <= rcode < 300:
--> 775     raise self.handle_error_response(
    776         rbody, rcode, resp.data, rheaders, stream_error=stream_error
    777     )
    778 return resp

InvalidRequestError: Unrecognized request argument supplied: functions

Did anyone find solution for this issue facing same

[georgiiic]

Hey ! Have you tried using Kor extraction_chain instead?

[georgiiic]

Given the error, I think the problem comes from the way you instantiate your model with Azure. I have never used it since I mostly use custom LLM or Opensource model from HugginFace, but I would advise you to check the parameters used, and if you havent made a typo.

As for the extraction chain, I would also advise you to dig a bit more and checked other ways to extract information with langchain, like Kor & Pydantic as it has helped me solve lots of errors.

[kamit-transient]

I tried the given snippet in question with together.ai's "mistralai/Mixtral-8x7B-Instruct-v0.1" model and it throws the error:

NOTE: as per together.ai docs give model does support the function and its compatible with OpenAI Api Docs

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File [/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain/output_parsers/openai_functions.py:72](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain/output_parsers/openai_functions.py:72), in JsonOutputFunctionsParser.parse_result(self, result, partial)
     [71](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain/output_parsers/openai_functions.py:71) try:
---> [72](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain/output_parsers/openai_functions.py:72)     function_call = message.additional_kwargs["function_call"]
     [73](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain/output_parsers/openai_functions.py:73) except KeyError as exc:

KeyError: 'function_call'

During handling of the above exception, another exception occurred:

OutputParserException                     Traceback (most recent call last)
Cell In[31], [line 20](vscode-notebook-cell:?execution_count=31&line=20)
     [15](vscode-notebook-cell:?execution_count=31&line=15) # llm = Replicate(
     [16](vscode-notebook-cell:?execution_count=31&line=16) #     model="a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5",
     [17](vscode-notebook-cell:?execution_count=31&line=17) #     input={"temperature": 0.75, "max_length": 500, "top_p": 1},
     [18](vscode-notebook-cell:?execution_count=31&line=18) # )
     [19](vscode-notebook-cell:?execution_count=31&line=19) chain = create_extraction_chain(schema=schema1, llm=llm)
---> [20](vscode-notebook-cell:?execution_count=31&line=20) chain.run(inp);

File [/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:145](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:145), in deprecated.<locals>.deprecate.<locals>.warning_emitting_wrapper(*args, **kwargs)
    [143](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:143)     warned = True
    [144](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:144)     emit_warning()
--> [145](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:145) return wrapped(*args, **kwargs)
...
---> [77](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain/output_parsers/openai_functions.py:77)         raise OutputParserException(f"Could not parse function call: {exc}")
     [78](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain/output_parsers/openai_functions.py:78) try:
     [79](https://file+.vscode-resource.vscode-cdn.net/opt/anaconda3/envs/langchain1/lib/python3.9/site-packages/langchain/output_parsers/openai_functions.py:79)     if partial:

OutputParserException: Could not parse function call: 'function_call'

[alibabadoufu]

I am having the same issue. Anyone found a solution especially when we are using the custom LLM template to create the LLM instance?

[georgiiic]

I am having the same issue. Anyone found a solution especially when we are using the custom LLM template to create the LLM instance?

[eyurtsev]

Extraction use case docs have been revamped: https://python.langchain.com/docs/use_cases/extraction

Please see documentation there with recommendations on how to extraction with LangChain.

[arad2022]

Year 2023 Same Question.

Code

repo_id="mistralai/Mistral-7B-Instruct-v0.1"

    llm = HuggingFaceHub(repo_id=repo_id)

    class Properties(BaseModel):
        person_name: str
        person_height: int
        person_hair_color: str
        dog_breed: Optional[str]
        dog_name: Optional[str]

    schema = {
    "properties": {
        "person_name": {"type": "string"},
        "person_height": {"type": "integer"},
        "person_hair_color": {"type": "string"},
        "dog_name": {"type": "string"},
        "dog_breed": {"type": "string"},
        "dog_extra_info": {"type": "string"},
    },
}
chain = create_extraction_chain_pydantic(pydantic_schema=Properties, llm=llm)

inp = """Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde."""

chain.run(inp)

ISSUE

raise OutputParserException( langchain_core.exceptions.OutputParserException: This output parser can only be used with a chat generation.

Did you find any solution or other way for using LLM from Hugging Face?

[arad2022]

I am having the same issue. Anyone found a solution especially when we are using the custom LLM template to create the LLM instance?

Did you find any solution for using other LLMs like Mistal?

[arieltoledo]

I tried to go with Kor, but what I'm building is in Spanish, so I need to change a lot of features to make it work. So I went for prompt engineering instead and found a flawless prompt for extractions, using mistral 7b 1.0

[georgiiic]

I did manage to make it work with mistral models too. I went for the Kor extraction chain and it worked fine.

[sanae-a11y]

Hi @georgiiic , please how did you do for the kor extraction using mistral .. because i try so without any solution .. thank you in advance
I got this problem :

[mtx-z]

@georgiiic Hi,

could you provide some examples of integration of Langchain + external LLM (eg huggingfaceHub/Endpoint) + Kor extraction?

Thank you

[georgiiic]

@sanae-a11y @mtx-z In this precise example, I used Vigogne (since it was french text), and here is how I made it work in a google colab environnement. But it should work with most text generation model.

# Imports
import langchain
import json
from kor.extraction import create_extraction_chain
from kor.nodes import Object, Text, Number
from langchain.llms import HuggingFacePipeline
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline, AutoModelForSeq2SeqLM

schema = Object(
    id="Person",
    description="Comment vous décririez une personne",
    attributes=[
        Text(
            id="First name",
            description="Prénom de la personne",
            examples=[],
            many=False,
        ),
        Text(
            id="Last name",
            description="Nom de famille de la personne",
            examples=[],
            many=False,
        ),
    ],
    many=True,
)

# Model
tokenizer = AutoTokenizer.from_pretrained("vigogne/language-model")
model = AutoModelForCausalLM.from_pretrained("vigogne/language-model")

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    max_length=4096
)

local_llm = HuggingFacePipeline(pipeline=pipe)

chain = create_extraction_chain(local_llm, schema, encoder_or_encoder_class="JSON")

# Extraction
persons = chain.invoke("your text here")

print(persons['text']['raw'])

[sanae-a11y]

Hi @georgiiic thank you so much for your help ! it's working now 💯

[ShkAmmarHussain]

I am using GoogleGenAI's GeminiPro as an LLM to get a structured output but when i try to invoke the chain it returns the following error that None type doesn't have an attribute get. Can anyone help me with this?

CODE: table_details_prompt = f"""Return the names of ALL the SQL tables that MIGHT be relevant to the user question. \ The tables are:

{table_details}

Remember to include ALL POTENTIALLY RELEVANT tables, even if you're not sure that they're needed."""

table_chain = create_extraction_chain_pydantic(Table, llm, system_message=table_details_prompt) tables = table_chain.invoke({"input": "give me info about the driver"})

Error: File c:\Users\Lenovo\anaconda3\envs\fms-chat\lib\site-packages\langchain_core\runnables\base.py:2499, in RunnableSequence.invoke(self, input, config) 2497 try: 2498 for i, step in enumerate(self.steps): -> 2499 input = step.invoke( 2500 input, 2501 # mark each step as a child run 2502 patch_config( 2503 config, callbacks=run_manager.get_child(f"seq:step:{i+1}") 2504 ), 2505 ) ... ---> 62 schema = first_function_declaration.get("parameters") 63 if schema is None or isinstance(schema, glm.Schema): 64 return tool

AttributeError: 'NoneType' object has no attribute 'get'