neo4j-labs / llm-graph-builder

Neo4j graph construction from unstructured data
Apache License 2.0
369 stars 93 forks source link

Integrate structured data as well for QnA in addition to unstructured data from pdf files #379

Open swanandbagve opened 1 month ago

swanandbagve commented 1 month ago

can u suggest what should be steps to integrate structured data along with unstructured data from pdf files so that we can get answers for questions that combined data from structured and unstructured data sources both ? looking forward to knowing which functions i need to modify . I think I will have to call function update_embedding_create_vector_index( graph, chunkId_chunkDoc_list, file_name) with each row from the structured data set ? and create one embedding column that creates embedding for all columns combined for that instance of the entity /node?

For instance if I have a Person table and personname, city, state, Age, Occupation and 50 other such columns will I need to create 1 embedding value for each person and store it as a separate property in neo4j ?

Just wondering what all will I need to modify for the QnA code to answer questions combing structured and unstructured data both

jexp commented 1 month ago

You can integrate your structured data separately e.g. with the data importer tool (which currently supports CSV) or by manually importing structured data into the graph.

https://neo4j.com/docs/data-importer/current/

swanandbagve commented 1 month ago

Hi @jexp , i already imported structured data using cypher load csv as i believe data importer is only available for Aura workspace or as a web tool and not part of desktop. However the problem is since this tool is retrieving data only using embedding similarity search despite creating relationships i am unable to retrieve the structured data using the chat interface of this tool. So my original question 'Just wondering what all will I need to modify for the QnA code to answer questions combing structured and unstructured data both' is still open

jexp commented 3 weeks ago

Data importer/workspace can also connect to local neo4j instances.

Good point on integrating the structured data, we'll think about it.