zilliztech / GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
https://gptcache.readthedocs.io
MIT License
6.96k stars 490 forks source link

[Bug]: Cannot start GPT cache server with docker #545

Closed julie1234567890 closed 9 months ago

julie1234567890 commented 10 months ago

Current Behavior

When

$ docker run -p 8000:8000 -it zilliz/gptcache:latest

I got

Downloading model.onnx: 100%|██████████████████████████████████████████████████████| 46.9M/46.9M [00:08<00:00, 5.85MB/s]
Traceback (most recent call last):
  File "/usr/local/bin/gptcache_server", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/gptcache_server/server.py", line 181, in main
    init_similar_cache(args.cache_dir)
  File "/usr/local/lib/python3.8/site-packages/gptcache/adapter/api.py", line 174, in init_similar_cache
    embedding = Onnx()
  File "/usr/local/lib/python3.8/site-packages/gptcache/embedding/__init__.py", line 52, in Onnx
    return onnx.Onnx(model)
  File "/usr/local/lib/python3.8/site-packages/gptcache/embedding/onnx.py", line 37, in __init__
    self.ort_session = onnxruntime.InferenceSession(onnx_model_path)
  File "/usr/local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 432, in __init__
    raise e
  File "/usr/local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/usr/local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 451, in _create_inference_session
    raise ValueError(
ValueError: This ORT build has ['AzureExecutionProvider', 'CPUExecutionProvider'] enabled. Since ORT 1.9, you are required to explicitly set the providers parameter when instantiating InferenceSession. For example, onnxruntime.InferenceSession(..., providers=['AzureExecutionProvider', 'CPUExecutionProvider'], ...)

Should it be a issue should fixed by push new image?

Expected Behavior

No response

Steps To Reproduce

No response

Environment

Windows, Docker: Linux container

Anything else?

No response

SimFG commented 10 months ago

@julie1234567890 i will release a new version today

ShaotuJia commented 10 months ago

Same problem for me.

rsharath commented 9 months ago

Hey Folks - was this issue resolved? I am still getting the same error message: Traceback (most recent call last): gptcache | File "/usr/local/bin/gptcache_server", line 8, in gptcache | sys.exit(main()) gptcache | File "/usr/local/lib/python3.8/site-packages/gptcache_server/server.py", line 181, in main gptcache | init_similar_cache(args.cache_dir) gptcache | File "/usr/local/lib/python3.8/site-packages/gptcache/adapter/api.py", line 174, in init_similar_cache gptcache | embedding = Onnx() gptcache | File "/usr/local/lib/python3.8/site-packages/gptcache/embedding/init.py", line 52, in Onnx gptcache | return onnx.Onnx(model) gptcache | File "/usr/local/lib/python3.8/site-packages/gptcache/embedding/onnx.py", line 37, in init gptcache | self.ort_session = onnxruntime.InferenceSession(onnx_model_path) gptcache | File "/usr/local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 432, in init gptcache | raise e gptcache | File "/usr/local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in init gptcache | self._create_inference_session(providers, provider_options, disabled_optimizers) gptcache | File "/usr/local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 451, in _create_inference_session gptcache | raise ValueError( gptcache | ValueError: This ORT build has ['AzureExecutionProvider', 'CPUExecutionProvider'] enabled. Since ORT 1.9, you are required to explicitly set the providers parameter when instantiating InferenceSession. For example, onnxruntime.InferenceSession(..., providers=['AzureExecutionProvider', 'CPUExecutionProvider'], ...) gptcache exited with code 1

Using zilliz/gptcache:latest

SimFG commented 9 months ago

@rsharath you can try to update the latest image, which is v0.1.42.