search

TheR1D / shell_gpt

A command-line productivity tool powered by AI large language models like GPT-4, will help you accomplish your tasks faster and more efficiently.
MIT License
8.84k stars 696 forks source link

Ollama integration 🦙 #461

Closed TheR1D closed 4 months ago

TheR1D commented 5 months ago

Implement an option to integrate Ollama with ShellGPT. These changes should include the ability to easily switch LLM backends for ShellGPT, allowing users to toggle between OpenAI and Ollama. Since Ollama responses are slightly different (compared to OpenAI), we can utilise Ollama Python Library.

With multiple LLM backends, dependencies specific to a particular LLM/Backend should utilize the Python package 'extras'. This way, users will have the option to install ShellGPT with the default OpenAI client, e.g., pip install shell-gpt, or with a specific backend, e.g., pip install shell-gpt[ollama]

Since Ollama supports multiple open-source models, we need to identify the specific model that performs best for ShellGPT use cases. Based on my research, mistral:7b-instruct outperforms ollama2:*-* in shell command generation tasks.

UPD: Seems it would be much easier to integrate Ollama using LiteLLM. This will also enable ShellGPT to work with the Azure OpenAI API.

rcspam commented 5 months ago

Bravo.... It could be cool to integrate mistralai api too.

TheR1D commented 5 months ago

I would greatly appreciate any assistance with testing the Ollama Integration PR #463, especially on WSL environments.