bug or missing doc: parsed with `AsFtSearch` contains zero `Score` value when working with vector search

[nekomeowww]

Summary

I was doing some research on Redis and vector search capabilities for semantic cache, but I found myself lost due to the emptiness of FtSearchDoc.Score as 0 value.

When executing such command:

func (c *RueidisJSON) RetrieveByVectors(ctx context.Context, vectors []float64, first int) (error) {
    cmd := c.rueidis.B().
        FtSearch().Index(c.repo.IndexName()).Query("(*)=>[KNN "+strconv.FormatInt(int64(first), 10)+" @vec $V]").
        Params().Nargs(2).NameValue().NameValue("V", rueidis.VectorString64(vectors)).
        Dialect(2).
        Build()

    resp := c.rueidis.Do(ctx, cmd)

    _, records, err := resp.AsFtSearch()
    if err != nil {
        return nil, err
    }

    xo.PrintJSON(records)      
}

the Score field of records[int] will be zero all the time due to the (maybe missing or incorrectly handled) field parsing strategy over implementation of:

https://github.com/redis/rueidis/blob/55e08314a4e11d376202acfd30e70f8012199203/message.go#L1046-L1073

I discovered this when debugging the redis messages:

As shown above, the __vec_score is present as child of extra_attributes instead of results when working with vectors, which lead the Score field to be unassigned.

I haven't tested yet, but I suspect the field that indicates the score (__vec_score in this case) is not always in the right position for other types of FT.SEARCH command, therefore the parsing of score under result is intended for other types of FT.SEARCH command, then this would not be an implementation issue.

Proposal

• How about introducing an AsFtSearchVector() method for processing the vectors correctly?
• How about adding a fallback strategy to read the __vec_score in extra_attributes?

Addition notes

For the code I provided above, I went through the:

• 386

• 413

along with the official documentation:

• FT.SEARCH | Docs
• Redis as a vector database quick start guide | Docs

TBH, the documentation of __vec_score rarely exists, and typos.

Current workaround

records = lo.Map(records, func(record rueidis.FtSearchDoc, _ int) rueidis.FtSearchDoc {
    if record.Score != 0 {
        return record
    }

    record.Score, _ = strconv.ParseFloat(record.Doc["__vec_score"], 64)
    return record
})

[rueian]

Hi @nekomeowww,

I believe the Score field is populated when using FT.SEARCH ... WITHSCORES, but I am not sure if the score will be the __vec_score in this case.

[nekomeowww]

Thanks for explaining, I went to check the Score field, it indeed returned, but with 0 as value and its value differs from __vec_score:

I will go and check what does Score mean and adjust my implementation, thanks.