zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
https://gptcache.readthedocs.io
MIT License
6.89k stars 480 forks source link

[DOCS]: similarity_threshold documentation is wrong #578

Open mayalinetsky-kryon opened 7 months ago

mayalinetsky-kryon commented 7 months ago

Documentation Link

https://gptcache.readthedocs.io/en/latest/references/gptcache.html?highlight=config#module-gptcache.config

Describe the problem

The documentation says:

similarity_threshold (float) – a threshold ranged from 0 to 1 to filter search results with similarity score higher than the threshold. When it is 0, there is no hits. When it is 1, all search results will be returned as hits.

But when I initialize the cache like so:

cache_base = CacheBase("sqlite")
vector_base = VectorBase("faiss", dimension=embedding_onnx.dimension)
data_manager = get_data_manager(cache_base, vector_base, max_size=100000)
cache.init(
    embedding_func=embedding_onnx.to_embeddings,
    data_manager=data_manager,
    similarity_evaluation=SearchDistanceEvaluation(max_distance=1),
    config=Config(similarity_threshold=1),
)

I don't get any hits.

Describe the improvement

Reverse the description of similarity_threshold.

Anything else?

Also, the syntax of "filter search results with similarity score higher than the threshold" is not defined enough. Are we filtering OUT results with a similarity higher than the threshold, and discarding them? or are we looking ONLY at items with similarity higher than the threshold?