zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

https://gptcache.readthedocs.io

MIT License

6.89k stars 480 forks source link

[Bug]: RuntimeError: The size of tensor a (2467) must match the size of tensor b (512) at non-singleton dimension 1 #584

Closed sdd031215 closed 7 months ago

sdd031215 commented 7 months ago

Current Behavior

cache_embeddings = Huggingface("model/bge-large-zh-v1.5") def get_contentfunc(data, **): return data.get("prompt").split("Question")[-1]

def init_gptcache_map(cache_obj: Cache): cache_base = CacheBase('sqlite') vector_base = VectorBase('milvus', host='xx', port='19530', dimension=1024, collection_name='chatbot') data_manager = get_data_manager(cache_base, vector_base) cache_obj.init( pre_embedding_func=get_content_func, embedding_func=cache_embeddings.to_embeddings, data_manager=data_manager, similarity_evaluation=SearchDistanceEvaluation(),
) langchain.llm_cache = GPTCache(init_gptcache_map)

Expected Behavior

I need your help , thanks. @SimFG @technicalpickles @jmahmood @shanghaikid @leio10 @cxie

Steps To Reproduce

No response

Environment

No response

Anything else?

No response

SimFG commented 7 months ago

@sdd031215 The same code, has the embedding function been modified in the last running?

sdd031215 commented 7 months ago

@sdd031215 The same code, has the embedding function been modified in the last running?

sdd031215 commented 7 months ago

cache_base = CacheBase('sqlite') vector_base = VectorBase('milvus', host='xx', port='19530', dimension=1024, collection_name='chatbot') data_manager = get_data_manager(cache_base, vector_base, max_size=100000)

init_similar_cache( pre_func=get_prompt, cache_obj=cache,
embedding=cache_embeddings, data_manager=data_manager,

evaluation=SearchDistanceEvaluation(max_distance=0.5, positive=True)

This method can be executed, but it is not written in sqlite and milvus. Please help me figure out why

SimFG commented 7 months ago

I probably guessed the reason, it's because the dimensions are set wrong. You can confirm whether the model you are using is from 1024dim.

sdd031215 commented 7 months ago

I probably guessed the reason, it's because the dimensions are set wrong. You can confirm whether the model you are using is from 1024dim.

i confirm. model from https://huggingface.co/BAAI/bge-large-zh-v1.5

sdd031215 commented 7 months ago

cache_base = CacheBase('sqlite') vector_base = VectorBase('milvus', host='xx', port='19530', dimension=1024, collection_name='chatbot') data_manager = get_data_manager(cache_base, vector_base, max_size=100000)

init_similar_cache( pre_func=get_prompt, cache_obj=cache, embedding=cache_embeddings, data_manager=data_manager, # evaluation=SearchDistanceEvaluation(max_distance=0.5, positive=True) )

This method can be executed, but it is not written in sqlite and milvus. Please help me figure out why

Is there anything missing in this method? @SimFG

SimFG commented 7 months ago

@sdd031215 no, Because it is an initialization step, no data will be written