how to index a table with 4096 dimension vectors?

[legaltextai]

Is it possible? the default indexing with hnsw does not permit more than 2k dimensions. thank you

[olirice]

an indexing maximum of 2k dimensions is a restriction of pgvector

for performance reasons you most likely would not want embeddings of that size In some cases, like OpenAI models you can make use of them being matryoshka embeddings to reduce the dimensionality with very little loss of accuracy

Here's a link to the blog post about that

[legaltextai]

Thank you Oliver