TypeError when used with xusenlinzy/api-for-open-llm

TheR1D / shell_gpt

A command-line productivity tool powered by AI large language models like GPT-4, will help you accomplish your tasks faster and more efficiently.

MIT License

8.86k stars 697 forks source link

TypeError when used with xusenlinzy/api-for-open-llm #411

Closed pengtb closed 4 months ago

pengtb commented 6 months ago

xusenlinzy/api-for-open-llm is another backend similar to LocalAI that can provide OpenAI-like API for local LLMs. However it is wierd that its generated completions end with a None, then shell_gpt would show a TypeError. Maybe better to add type checks before concatenating? Places are:

sgpt/cache.py:40
sgpt/handlers/handler.py:37

timcrowell commented 6 months ago

I'm seeing the exact same thing when using BerriAI/litellm as the backend. Litellm would be particularly useful in combination with shell_gpt because it would allow switching between a local model and OpenAI using just the --model switch.

TheR1D commented 6 months ago

I didn't test sgpt with any other backends except OpenAI and LocalAI, but as I can see the error you are getting is related to cache. Try to disable caching sgpt --no-cache "test".

TheR1D commented 4 months ago

ShellGPT with LiteLLM was released as 1.3.0. This should fix the issue, given that I'm closing this issue, but feel free to reopen it.