Ollama LLM does not work with LLMGraphTransformer

[ali-habibzadeh]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain.js documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain.js rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

const gpt = new ChatOpenAI({ temperature: 0, model: "gpt-4o", apiKey: appConfig.OPEN_AI_API_KEY });
const ollama = new Ollama({ baseUrl: "http://localhost:11434", model: "llama3", format: "json" });

export const getGraphTransformer = () => {
  return new LLMGraphTransformer({
    llm: ollama,
    allowedNodes: nodes,
    allowedRelationships: relationships,
    strictMode: false,
  });
};

Error Message and Stack Trace (if applicable)

llm.cjs:149
            throw new Error("The specified LLM does not support the 'withStructuredOutput'. Please ensure you are using an LLM that supports this feature.");
                  ^
source-map-support.js:722
Error: The specified LLM does not support the 'withStructuredOutput'. Please ensure you are using an LLM that supports this feature.

Description

It's not possible to drop in Llama instead of OpenAI with LLMGraphTransformer.

System Info

"@langchain/community": "^0.2.18",
"@langchain/openai": "^0.2.1",
"langchain": "^0.2.9",

[dosubot[bot]]

The error you're encountering is due to the Ollama LLM not supporting the withStructuredOutput method, which is required by the LLMGraphTransformer class. To resolve this issue, you need to use an LLM that supports the withStructuredOutput method.

Here are some LLMs in LangChain.js that support the withStructuredOutput method and can be used as a replacement for Ollama:

• Anthropic
• Google VertexAI
• Mistral
• OpenAI
• Groq ^[1][2].

You can switch to one of these models in your LLMGraphTransformer configuration. For example, using OpenAI:

const gpt = new ChatOpenAI({ temperature: 0, model: "gpt-4-turbo", apiKey: appConfig.OPEN_AI_API_KEY });

export const getGraphTransformer = () => {
  return new LLMGraphTransformer({
    llm: gpt,
    allowedNodes: nodes,
    allowedRelationships: relationships,
    strictMode: false,
  });
};

This should resolve the error and allow you to use the LLMGraphTransformer with a compatible LLM.

To continue talking to Dosu, mention @dosu.

[ali-habibzadeh]

@dosu can you generate an example using Groq?

[ali-habibzadeh]

Tried the below and its fails every time:

import { ChatGroq } from "@langchain/groq";
import { LLMGraphTransformer } from "@langchain/community/experimental/graph_transformers/llm";

const groq = new ChatGroq({ apiKey: appConfig.GROQ_API_KEY, temperature: 0 });

export const getGraphTransformer = () => {
  return new LLMGraphTransformer({
    llm: groq,
    allowedNodes: nodes,
    allowedRelationships: relationships,
    strictMode: false,
  });
};

Error:

Reclaiming failed request back to the list or queue. 400 {"error":{"message":"Failed to call a function. Please adjust your prompt. See 'failed_generation' for more details.","type":"invalid_request_error","code":"tool_use_failed","failed_generation":"{\n\t\"tool_call\": {\n\t\t\"id\": \"pending\",\n\t\t\"type\": \"function\",\n\t\t\"function\": {\n\t\t\t\"name\": \"extract\"\n\t\t},\n\t\t\"parameters\": {\n\t\t\t\"nodes\": [\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Netflix Tudum - Go behind the streams\",\n\t\t\t\t\t\"type\": \"WEB_PAGE\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Thursday Murder Club\",\n\t\t\t\t\t\"type\": \"TV_SHOW\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Richard Osman’s novel\",\n\t\t\t\t\t\"type\": \"NOVEL\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"John Dilillo\",\n\t\t\t\t\t\"type\": \"JOURNALIST\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"3:00 PM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"BURNING QUESTIONS\",\n\t\t\t\t\t\"type\": \"SECTION_HEADING\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"What Does It Take to Find the Perfect Mole?\",\n\t\t\t\t\t\"type\": \"QUESTION\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"It’s like working on a Ferrari. (We’ll explain that, promise.)\",\n\t\t\t\t\t\"type\": \"STATEMENT\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"BY JEAN BENTLEY\",\n\t\t\t\t\t\"type\": \"AUTHOR\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"2:00 PM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"CASTING CALL\",\n\t\t\t\t\t\"type\": \"SECTION_HEADING\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Watch More Midnight Movies from The Mole Season 2\",\n\t\t\t\t\t\"type\": \"INSTRUCTION\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Who speaks five languages?\",\n\t\t\t\t\t\"type\": \"QUESTION\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Who else is an undercover cop?\",\n\t\t\t\t\t\"type\": \"QUESTION\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"2:00 PM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"NEWS\",\n\t\t\t\t\t\"type\": \"SECTION_HEADING\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"In New Sports Doc Series Receiver, Fumbling Is Not an Option\",\n\t\t\t\t\t\"type\": \"STATEMENT\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"The new series from the creators of Quarterback premieres in summer 2024.\",\n\t\t\t\t\t\"type\": \"STATEMENT\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"BY WILLIAM EMMERSHY\",\n\t\t\t\t\t\"type\": \"AUTHOR\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"11:00 AM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Mia McKenna-Bruce\",\n\t\t\t\t\t\"type\": \"ACTOR\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"The Seven Dials Mystery\",\n\t\t\t\t\t\"type\": \"MOVIE\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"The thrilling new Agatha Christie adaptation is in production now.\",\n\t\t\t\t\t\"type\": \"STATEMENT\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"BY TARA BITRAN\",\n\t\t\t\t\t\"type\": \"AUTHOR\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"6:30 PM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Godcat\",\n\t\t\t\t\t\"type\": \"CHARACTER\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Reaches Epic Purr-Portions in Exploding Kittens\",\n\t\t\t\t\t\"type\": \"STATEMENT\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"6:00 PM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"All Rise: Suits Season 9\",\n\t\t\t\t\t\"type\": \"SECTION_HEADING\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"4:30 PM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"The Pods Are Heading Across the Pond for Love Is Blind: UK\",\n\t\t\t\t\t\"type\": \"STATEMENT\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"YESTERDAY 7:58 PM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"Everything We Know About Vikings: Valhalla’s Third and Final Season\",\n\t\t\t\t\t\"type\": \"STATEMENT\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"YESTERDAY 7:01 PM\",\n\t\t\t\t\t\"type\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"id\": \"MORE NEWS\",\n\t\t\t\t\t\"type\": \"SECTION_HEADING\"\n\t\t\t\t}\n\t\t\t],\n\t\t\t\"relationships\": [\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"PUBLISHED\",\n\t\t\t\t\t\"sourceNodeId\": \"John Dilillo\",\n\t\t\t\t\t\"sourceNodeType\": \"JOURNALIST\",\n\t\t\t\t\t\"targetNodeId\": \"Netflix Tudum - Go behind the streams\",\n\t\t\t\t\t\"targetNodeType\": \"WEB_PAGE\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"WRITTEN_BY\",\n\t\t\t\t\t\"sourceNodeId\": \"Thursday Murder Club\",\n\t\t\t\t\t\"sourceNodeType\": \"TV_SHOW\",\n\t\t\t\t\t\"targetNodeId\": \"Richard Osman’s novel\",\n\t\t\t\t\t\"targetNodeType\": \"NOVEL\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"WRITTEN_BY\",\n\t\t\t\t\t\"sourceNodeId\": \"Thursday Murder Club\",\n\t\t\t\t\t\"sourceNodeType\": \"TV_SHOW\",\n\t\t\t\t\t\"targetNodeId\": \"John Dilillo\",\n\t\t\t\t\t\"targetNodeType\": \"JOURNALIST\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"What Does It Take to Find the Perfect Mole?\",\n\t\t\t\t\t\"sourceNodeType\": \"QUESTION\",\n\t\t\t\t\t\"targetNodeId\": \"BURNING QUESTIONS\",\n\t\t\t\t\t\"targetNodeType\": \"SECTION_HEADING\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ANSWERS\",\n\t\t\t\t\t\"sourceNodeId\": \"It’s like working on a Ferrari. (We’ll explain that, promise.)\",\n\t\t\t\t\t\"sourceNodeType\": \"STATEMENT\",\n\t\t\t\t\t\"targetNodeId\": \"What Does It Take to Find the Perfect Mole?\",\n\t\t\t\t\t\"targetNodeType\": \"QUESTION\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"WRITTEN_BY\",\n\t\t\t\t\t\"sourceNodeId\": \"What Does It Take to Find the Perfect Mole?\",\n\t\t\t\t\t\"sourceNodeType\": \"QUESTION\",\n\t\t\t\t\t\"targetNodeId\": \"Jean Bentley\",\n\t\t\t\t\t\"targetNodeType\": \"JOURNALIST\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"Who speaks five languages?\",\n\t\t\t\t\t\"sourceNodeType\": \"QUESTION\",\n\t\t\t\t\t\"targetNodeId\": \"CASTING CALL\",\n\t\t\t\t\t\"targetNodeType\": \"SECTION_HEADING\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"Who else is an undercover cop?\",\n\t\t\t\t\t\"sourceNodeType\": \"QUESTION\",\n\t\t\t\t\t\"targetNodeId\": \"CASTING CALL\",\n\t\t\t\t\t\"targetNodeType\": \"SECTION_HEADING\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"In New Sports Doc Series Receiver, Fumbling Is Not an Option\",\n\t\t\t\t\t\"sourceNodeType\": \"STATEMENT\",\n\t\t\t\t\t\"targetNodeId\": \"NEWS\",\n\t\t\t\t\t\"targetNodeType\": \"SECTION_HEADING\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"WRITTEN_BY\",\n\t\t\t\t\t\"sourceNodeId\": \"In New Sports Doc Series Receiver, Fumbling Is Not an Option\",\n\t\t\t\t\t\"sourceNodeType\": \"STATEMENT\",\n\t\t\t\t\t\"targetNodeId\": \"William Emmershy\",\n\t\t\t\t\t\"targetNodeType\": \"JOURNALIST\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"Mia McKenna-Bruce\",\n\t\t\t\t\t\"sourceNodeType\": \"ACTOR\",\n\t\t\t\t\t\"targetNodeId\": \"The Seven Dials Mystery\",\n\t\t\t\t\t\"targetNodeType\": \"MOVIE\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"The Seven Dials Mystery\",\n\t\t\t\t\t\"sourceNodeType\": \"MOVIE\",\n\t\t\t\t\t\"targetNodeId\": \"Tara Bitran\",\n\t\t\t\t\t\"targetNodeType\": \"JOURNALIST\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"Godcat\",\n\t\t\t\t\t\"sourceNodeType\": \"CHARACTER\",\n\t\t\t\t\t\"targetNodeId\": \"Reaches Epic Purr-Portions in Exploding Kittens\",\n\t\t\t\t\t\"targetNodeType\": \"STATEMENT\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"All Rise: Suits Season 9\",\n\t\t\t\t\t\"sourceNodeType\": \"SECTION_HEADING\",\n\t\t\t\t\t\"targetNodeId\": \"4:30 PM\",\n\t\t\t\t\t\"targetNodeType\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"The Pods Are Heading Across the Pond for Love Is Blind: UK\",\n\t\t\t\t\t\"sourceNodeType\": \"STATEMENT\",\n\t\t\t\t\t\"targetNodeId\": \"YESTERDAY 7:58 PM\",\n\t\t\t\t\t\"targetNodeType\": \"TIME\"\n\t\t\t\t},\n\t\t\t\t{\n\t\t\t\t\t\"relationshipType\": \"ABOUT\",\n\t\t\t\t\t\"sourceNodeId\": \"Everything We Know About Vikings: Valhalla’s Third and Final Season\",\n\t\t\t\t\t\"sourceNodeType\": \"STATEMENT\",\n\t\t\t\t\t\"targetNodeId\": \"YESTERDAY 7:01 PM\",\n\t\t\t\t\t\"targetNodeType\": \"TIME\"\n\t\t\t\t}\n\t\t\t]\n\t\t}\n\t}\n}"}}

This error occurs when I use any of Groq models. I tried all of them except whisper of course

[csaiedu]

I am able to run the code below fine on Ollama. Apologizes if it's in python, but it's a proof of concept. ChatOllama has a withStructuredOutput function working, but what fails in ChatOllama is the node_properties and relationship_properties parameters when set. Also just make sure you have the newest version of langchain_experimental

from langchain_experimental.graph_transformers import LLMGraphTransformer
from langchain_community.chat_models import ChatOllama

llm = ChatOllama(
model="llama3",
format='json'
)  

llm_transformer = LLMGraphTransformer(
    llm=llm,
    allowed_nodes=['Category'],
    allowed_relationships=['Relation'],
    #node_properties=["animal"],
    #relationship_properties=["IS_A"],
    strict_mode=False
)

from langchain_core.documents import Document

llm_transformer.convert_to_graph_documents([Document(page_content="A cat is an animal. It has a tail")])

[cip22]

@csaiedu Thanks for the example. This is the way it's meant to be. In my case, the code is running without error, but the nodes and relationships lists are empty.

With ChatOpenai instead of ChatOllama, the result looks great:

nodes=[Node(id='Cat', type='Animal'), Node(id='Mat', type='Object'), Node(id='Tail', type='Body_part')] relationships=[Relationship(source=Node(id='Cat', type='Animal'), target=Node(id='Mat', type='Object'), type='LOCATION', properties={'description': 'on'}), Relationship(source=Node(id='Cat', type='Animal'), target=Node(id='Tail', type='Body_part'), type='HAS')] source=Document(page_content='The cat is on the mat. It has a tail')

I didn't thoroughly look into the code, but I guess that it depends on the way the functions are called by ChatOpenai resp. ChatOllama.