Question Setting ef_search to different values does not affect number of results retrieved (Django 4.2 + PGVector + HNSW index)

[rmincling]

Hi all,

I have this simple Django model setup:

class AiVector(models.Model):
    # Embeddings
    embedding_sbert = VectorField(dimensions=768, null=True, blank=True) # using SBERT embedding (all-mpnet-base-v2)

    class Meta:
        indexes = [
            # SBERT
            HnswIndex(
                name='vector_sbert_index',
                fields=['embedding_sbert'],
                m=16,
                ef_construction=64, 
                opclasses=['vector_cosine_ops']  
            )]

I am trying to do a basic vector search (cosine) on the model with a simple question/embedding with the following code:

         with connection.cursor() as cursor:
                cursor.execute("SET LOCAL hnsw.ef_search = 40")
            self.matching_vectors = AiVector.objects \
                .annotate(distance=CosineDistance("embedding_sbert", text_query_embedding)) \
                .filter(distance__lte=0.8)

No matter what i set ef_search to, whether it be 1, 40, 60, 100, I always get the same number of results. What is the correct way to set ef_search?

Regards,

Rob

[jkatz]

@rmincling How many results are you getting?

(FWIW, it's been awhile since I've written heavy Django code, but I suspect that you're executing the query using two different cursors, which means it's unaware of the setting of hnsw.ef_search that you're explicitly setting.)

[rmincling]

I'm getting over 3000 results

[jkatz]

@rmincling In that case, this is because you haven't set a LIMIT, and PostgreSQL is opting to use a table scan instead of an index scan. I suggest trying:

            self.matching_vectors = AiVector.objects \
                .annotate(distance=CosineDistance("embedding_sbert", text_query_embedding)) \
                .filter(distance__lte=0.8)[:10]

to return 10 results.

[rmincling]

I will try that. How do i check if the hnsw index is being used at all?

[jkatz]

@rmincling With the aforementioned disclaimer that I'm very rusty with Django, you can introspect the query plain with Queryset.explain().

That said, if your dataset is only 3,000 records, you're better off not using the index and simply doing a table scan, as timing wise that would be roughly comparable to using the index (the index scan may be a bit faster) and you'll achieve 100% recall. If the intention is for this workload to grow, I'd suggest testing with a larger set of vectors.

[rmincling]

ok, so here is an interesting update, regardless of what LIMIT I used, the HNSW index was NOT being used (confirmed when using EXPLAIN ANALYZE. It was using a sequential scan.

So I tried this:

cursor.execute("SET LOCAL enable_seqscan TO off;")

Resulting in the HNSW being used.

So now, when i change the value of ef_search the amount of results changes accordingly.

cursor.execute("SET LOCAL hnsw.ef_search = 100")

My next question is, I expect this vector table to have potentially 100k rows. So how do I decide if I should used HNSW index or not?

[jkatz]

I expect this vector table to have potentially 100k rows. So how do I decide if I should used HNSW index or not?

In this case, PostgreSQL will likely opt to use the HNSW index for your query. You shouldn't need to set enable_seqscan (and for a production workload I strongly recommend against doing so).

[rmincling]

It seems the maximum value of ef_search is 1000, didn't notice that before. Wonder will that be increased in future versions of postgres/pgvector

[jkatz]

@rmincling If you're interested in discusing hnsw.ef_search limits, I'd recommend opening up a new issue with the stated use case as to why it'd be beneficial to have it increased. If you're now unblocked with your original question, I'd recommend closing out this issue. Thanks!

[rmincling]

Ok, will do!, Thanks for all of your help and insights