Add the possibility to use local open-source models instead of APIs

[wilderlopes]

What is it?

As users, we would like to prevent data to be sent from our local environment to the cloud. Currently, miniogre uses primarily cloud API providers (OpenAI, OctoAI, Grok, Mistral). This prevents miniogre from being adopted in enterprises due to data security issues.

Proposed solution

We must implement a way to use a local, open-source model that can process the source code. Perhaps we can achieve this by integrating ollama into miniogre

[metcalfc]

+1 to making Ollama work. Its API is straight forward.

curl http://localhost:11434/api/chat -d '{
  "model": "mistral",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

And the python bindings are already written. https://github.com/ollama/ollama-python

ollama.chat(model='llama2', messages=[{'role': 'user', 'content': 'Why is the sky blue?'}])

I finally got things working with octoai but it was a bit of a pain. I've already got Ollama running and would happily have pulled a specific model.

[andrewssobral]

@metcalfc agree ! I created a PR for adding OLLAMA (and similar ones) to miniogre here https://github.com/ogre-run/miniogre/pull/10 However, I need to solve some problems before merging, it seems that only occurs on Apple Silicon.

[wilderlopes]

Implemented in #10 and already merged to main.