[Question]: indey query error: shapes (1792,) and (1024,) not aligned: 1792 (dim 0) != 1024 (dim 0)

[justinzyw]

Question Validation

• [X] I have searched both the documentation and discord for an answer.

Question

I was using embed_model = HuggingFaceEmbedding(model_name="TencentBAC/Conan-embedding-v1") and llm = DashScope(model_name=“qwen-turbo”) for vector index generation and query.

The index program firstly generates an index and stores to local file, then query program loads the index and queries it.

Not sure why, some how the query program throws error below:

File "C:\Software\Python\chatgpt\eac_llm\Lib\site-packages\llama_index\core\base\embeddings\base.py", line 62, in similarity product = np.dot(embedding1, embedding2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: shapes (1792,) and (1024,) not aligned: 1792 (dim 0) != 1024 (dim 0)

[dosubot[bot]]

I found a similar issue that might be relevant to your problem. It seems like you're encountering a dimensionality mismatch between the embeddings used for index creation and querying. A similar issue was discussed in an open thread where the solution was to ensure that the same embedding model is used for both constructing and querying the index. If you change the embedding model, you need to rebuild the index to match the new model's dimensionality ^[1].

In your case, it appears that the TencentBAC/Conan-embedding-v1 model and the DashScope model might be producing embeddings of different dimensions (1792 and 1024, respectively). You should verify the dimensionality of the embeddings generated by each model and ensure they match. If they don't, you'll need to either adjust the models you're using or rebuild the index with a consistent embedding model.

To continue talking to Dosu, mention @dosu.

[logan-markewich]

@justinzyw dosu is right

generally this error happens when you created an index with one embedding model, and queried it with another. You can change embedding models without first re-creating the index and re-embedding your data

[justinzyw]

@logan-markewich thanks for your comment! I was using the same embedding model (I will double-check my code on this though) and it seems to me that some embedding models use different dimensions at generation and query. Is it a possible reason? @dosu

[dosubot[bot]]

Yes, it is possible for some embedding models to use different dimensions at generation and query, even if the same model is used. For example, the jina-embeddings-v3 model supports Matryoshka Representation Learning, which allows users to control the embedding dimension with minimal performance loss. You can specify different dimensions for text and query embeddings, such as 1024 for text embeddings and 512 for query embeddings ^[1].

To continue talking to Dosu, mention @dosu.

[justinzyw]

@dosu, how to control dimension at index construction and query separately?

[dosubot[bot]]

To control the embedding dimensions separately during index construction and querying in the LlamaIndex library, you need to ensure that the embedding models you use for both processes support configurable dimensions. For instance, the HuggingFaceEmbedding model allows you to specify the dimensionality during initialization using the dimensionality parameter:

from llama_index.embeddings.huggingface import HuggingFaceEmbedding

# For index construction
embedding_model_construction = HuggingFaceEmbedding(
    model_name="your-model-name",
    dimensionality=1024  # Specify the desired dimensionality
)

# For querying
embedding_model_query = HuggingFaceEmbedding(
    model_name="your-model-name",
    dimensionality=512  # Specify a different dimensionality if needed
)

Ensure that the models you choose for both index construction and querying support this kind of configuration. If the models inherently produce embeddings of different dimensions, you may need to adjust your approach or select models that allow for consistent dimensionality settings.

To continue talking to Dosu, mention @dosu.

[justinzyw]

@dosu, parm dimensionality does not work in class HuggingFaceEmbedding, but I found another parm truncate_dim=1024. What is it for? can it work in my scenario?

[dosubot[bot]]

Could you provide more details about the specific version of the HuggingFaceEmbedding class you are using? This will help determine if the truncate_dim parameter is applicable in your scenario. Additionally, any documentation or examples you have regarding the truncate_dim parameter would be helpful to understand its intended use.

To continue talking to Dosu, mention @dosu.

[logan-markewich]

I was using the same embedding model (I will double-check my code on this though) and it seems to me that some embedding models use different dimensions at generation and query.

What model has different dimensions for documents vs queries? That shouldn't really be possible