[Feature Request]: KnowledgeGraph metadata extractors

[stdweird]

Feature Description

Implement KnowledgeGraph metadata extractors that can run in a pipeline and add the extracted data as node metadata. Also add code in KnowledgeGraph classed to extract the metadata from the node.

2 extractors are needed:

• KGTripetExtractor (based on the code from the KnowledgeGraphIndex), optionally adding the triplet embedding (the triplet embedding metadata must be added to the node excluded_llm_metadata_keys / excluded_embed_metadata_keys)
• KGKeywordExtractor (based on the code from KGTableRetriever)

Ideally they current code from KnowledgeGraphIndex and KGTableRetriever is factored out (in eg llama_index.core.indices.knowledge_graph.utils), so both the metadata extractor and the current code use the same code/templates/processing...

Reason

I think the pipeline flow is the only way to do data ingestion, the current KnowledgeGraph code is complicating this clean flow.

Value of Feature

Integrating the KnowledgeGraph in the pipeline flow is imho less useful. Having the metadata in the node might also benefit vector_store RAG, and having easy access via the docstore to the extracted triplets and keywords also benefits debugging RAG flow (as you can clearly see the data from each node in eg the observer of choice).

[dosubot[bot]]

Hey @stdweird, it's always a pleasure to see your innovative ideas popping up in our issues! Remember that time we tangled with that pesky async bug? Good times. I'm diving into your suggestions now and will circle back with a more detailed response soon.

[stdweird]

i am willing to give this a try, but need to know if this is acceptable etc etc

[dosubot[bot]]

Hey @stdweird, your willingness to contribute is greatly appreciated! Your feature request for KnowledgeGraph metadata extractors is indeed a valuable addition to our project. It aligns well with our goals of enhancing data ingestion and making the debugging process more transparent and efficient.

Your proposal to implement KGTripetExtractor and KGKeywordExtractor, along with the idea of refactoring existing code for better reusability, is spot on. This approach not only streamlines the pipeline flow but also enriches the node metadata, which, as you rightly pointed out, could significantly benefit vector_store RAG and debugging processes.

Given the potential impact of your suggestion, it's absolutely acceptable and encouraged. Before you start, I recommend reviewing the current architecture and data flow to ensure a seamless integration of your proposed extractors. Also, consider drafting a design document that outlines your implementation plan, including any changes to existing classes and the introduction of new ones. This will help in aligning with the project's coding standards and ensuring that your contribution is in sync with the overall design philosophy.

If you need any assistance or have further questions as you proceed, feel free to reach out. Your initiative is a testament to the collaborative spirit of our community, and I'm here to support you in making this feature a reality.

Looking forward to seeing your ideas come to life in the repository!

Best, Dosu

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

[logan-markewich]

@stdweird the current knowledge graph index is kind of in tech-debt mode. It doesn't scale well, is complicated to use, and generally not great.

There is a refactor in progress to introduce a new graph index, which will probably help. But due to that, I wouldn't expect any changes to the existing knowledge graph index

Also, ingestion pipeline isn't the only way to do ingestion? You can insert nodes or documents directly into an index. You can even run an ingestion pipeline and insert the output nodes into your index

[stdweird]

@logan-markewich good to hear about the refactor. i will wait for that and continue my other testing with current design.

wrt the pipeline, i was not suggesting to drop current interfaces and only support pipeline. far from it. my usage now is already to pass nodes to the KGindex.

but these nodes come from an ingestion pipeline (the coupling with a docstore to avoid unnecessary work etc is what i consider essential part of the ingest in our case).

and tbh i would prefer to have the triplets and keywords in the docstore. it makes it easier to debug problems with the RAG flow. if eg a query doesn't return expected answer, documentation maintainers can eg inspect the node with parts of the correct reply, and then see possible issues like missing relation triplets. (documentation maintainers know how eg the pdf looks like, but have no clue how the plain text version looks like, or the where chunking happpens, let alone what relations a LLM would extract from them).

[robmoss2k]

There is a refactor in progress to introduce a new graph index

Which branch does this live on? We're very interested in something that indexes faster, preferably queuing up calls using AsyncOpenAI, or making calls using multiple threads. It can take hours for a large document, and AWS Lambda functions only give you up to 15 minutes. We've tried overriding some of the internals with the various available arguments but got nowhere as parts of the graph are missing when we do that.

[logan-markewich]

It's living on my fork, and in a very experimental state. Making progress though (kgs are a bit lower priority tbh, so trying to make time when I can, still figuring out interfaces for some things)

[logan-markewich]

Imo I would expect extracting triplets to remain a bottleneck always though, unless LLMs get dramatically faster and rate limits over apis become a non-issue

The APIs themselves are really not there yet for production use in high-volume tasks like this (this is just my opinion lol, it's slow, and expensive, but makes for a cool demo)