search

darrenburns / elia

A snappy, keyboard-centric terminal user interface for interacting with large language models. Chat with ChatGPT, Claude, Llama 3, Phi 3, Mistral, Gemma and more.
Apache License 2.0
1.86k stars 115 forks source link

Setup for local ollama models #53

Closed nossidge closed 5 months ago

nossidge commented 5 months ago

I do not understand how to set this up with my local ollama model.

Following the "Running local models" section of the readme, it says "The location of the configuration file is noted at the bottom of the options window (ctrl+o)". This leads me to try to run the program first, and I will then be able to find the config, by pressing ctrl+o. But I can't seem to run the program at all without an API key.

Running elia gives me this error:

Did not find openai_api_key, please add an environment variable `OPENAI_API_KEY` which contains it

So then I run with a dummy environment variable: OPENAI_API_KEY=foo elia

But then pressing ctrl+o does nothing at all.

Here's a screenshot: Screenshot from 2024-05-25 22-37-15

How can I set up the config file for my local ollama mistral model?

Many thanks for your help.

darrenburns commented 5 months ago

You seem to be running a very old version - please run pipx upgrade elia-chat and let me know if you're still having issues

nossidge commented 5 months ago

Wow fast reply, thanks for the attentiveness, that's really cool!

But it seems like I'm already at the latest version 0.4.1: Screenshot from 2024-05-25 22-49-21

darrenburns commented 5 months ago

The latest version is 1.7.0 🙂

Maybe try a pipx uninstall elia-chat followed by an install?

nossidge commented 5 months ago

I'm still getting version 0.4.1 - I even tried re-installing pipx Screenshot from 2024-05-25 23-05-08 For clarity, I'm on Ubuntu 22.04.4 and Python 3.10.12, if that helps? I don't know much about pipx requirements, I first installed it just now for this project!

darrenburns commented 5 months ago

Does 'pipx install elia-chat==1.7.0' work?

There are a few troubleshooting steps in this guide too - https://pipx.pypa.io/stable/troubleshooting/

If that doesn't work, maybe you'll have more luck with installing rye and doing "rye install elia-chat"

m1chae1bx commented 5 months ago

I've figured out the problem. You need to install pipx while on at least python 3.11, and then run pipx install elia-chat==1.7.0. Although that only solves the problem of installing the latest version

m1chae1bx commented 5 months ago

It still cannot find the api key even if I have already set it in an environment variable

image
m1chae1bx commented 5 months ago

I got GPT-4o finally working but only if I run elia in the following way:

OPENAI_API_KEY=<api key here> elia

It doesn't work if I set it as an environment variable in a separate command.

darrenburns commented 5 months ago

You must not be exporting the environment variable correctly. How are you setting it?

m1chae1bx commented 5 months ago

You're right, my bad. I forgot to add the export keyword. I've only set it as OPENAI_API_KEY=<api key here> instead of export OPENAI_API_KEY=<api key here>. Thanks!

nossidge commented 5 months ago

Apparently pipx was using my system python, instead of my pyenv version. Luckily there's a way to fix it, using the PIPX_DEFAULT_PYTHON to point at the pyenv shim:

PIPX_DEFAULT_PYTHON=/home/paul/.pyenv/shims/python pipx install elia-chat==1.7.0

It's now working perfectly, with my local LLM!

Thanks for bearing with me, as this issue really wasn't your fault at all.

darrenburns commented 5 months ago

Great, glad you sorted it!

No worries :)