Open haozhuoyuan opened 1 month ago
The VectorStoreIndex
stores embeddings for the input text chunks, along with metadata such as document IDs and node IDs. This information can be viewed by querying the index or by inspecting the underlying vector store directly.
The command print(index.index_struct)
only outputs index_id
because index_struct
is a minimal representation of the index structure, primarily used for identification purposes. The detailed embeddings and other stored information are managed by the vector store backend and are not directly exposed through index_struct
.
To view the detailed information, you would typically interact with the vector store directly or use the querying capabilities provided by VectorStoreIndex
.
Example of querying the index:
query_engine = index.as_query_engine()
response = query_engine.query("What did the author do growing up?")
print(response)
Additionally, the index_struct
contains more than just the index_id
. It includes a nodes_dict
mapping node IDs to their respective embeddings, a doc_id_dict
mapping document IDs to their associated node IDs, and an embeddings_dict
which stores the actual embeddings. However, these details are not printed by default with print(index.index_struct)
.
Here is an example of the structure:
{
"index_struct": {
"__type__": "simple_dict",
"__data__": {
"index_id": "9343c0c6-8313-4549-a423-1348960b6958",
"summary": null,
"nodes_dict": {
"ddb890e7-82d5-4dc5-a950-b498c3ef2494": "ddb890e7-82d5-4dc5-a950-b498c3ef2494",
"5a03268d-b654-48a4-b3ea-b9620a5ec614": "5a03268d-b654-48a4-b3ea-b9620a5ec614",
"8d29350a-a932-4665-b436-a810e66def48": "8d29350a-a932-4665-b436-a810e66def48",
"6b151549-3bcc-4e11-a181-c1ac4dd34c8a": "6b151549-3bcc-4e11-a181-c1ac4dd34c8a",
"e2ac2af1-b59f-4c83-b365-f0205ba1f732": "e2ac2af1-b59f-4c83-b365-f0205ba1f732",
"307b2feb-5b0a-4b86-aec7-994074c94d29": "307b2feb-5b0a-4b86-aec7-994074c94d29"
},
"doc_id_dict": {
"707af1fd-62af-4693-93e1-01433b900aca": [
"ddb890e7-82d5-4dc5-a950-b498c3ef2494",
"5a03268d-b654-48a4-b3ea-b9620a5ec614",
"8d29350a-a932-4665-b436-a810e66def48",
"6b151549-3bcc-4e11-a181-c1ac4dd34c8a",
"e2ac2af1-b59f-4c83-b365-f0205ba1f732",
"307b2feb-5b0a-4b86-aec7-994074c94d29"
]
},
"embeddings_dict": {}
}
}
}
The VectorStoreIndex
class in the LlamaIndex framework is designed to work with an existing vector store, and it includes methods for embedding nodes, adding nodes to the index, and querying the index. The detailed information about the nodes and their embeddings is managed by the vector store and can be accessed through the vector store's API or by using the provided querying methods [1][2][3].
To continue talking to Dosu, mention @dosu.
Stroage is split between an indec struct, docstroe, and vector store. This is all contained in the storage context.
If you use a vectordb integration. Only the vector store is used (everything is thrown in there) to simplify storage
@logan-markewich Through vector store, if I want to delete some specific nodes by using the index.delete_nodes() method, how can I get the corresponding node id? I hope you can help me to answer it again
Question Validation
Question