search

cocktailpeanut / dalai

The simplest way to run LLaMA on your local machine
https://cocktailpeanut.github.io/dalai
13.09k stars 1.42k forks source link

Error with current Python #2

Open jooray opened 1 year ago

jooray commented 1 year ago

I have a Python 3.11 installed by default. It seems that pytorch does not work, this is the output of npx dalai llama:

pip3 install torch torchvision torchaudio sentencepiece numpy
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch

I think the reason is this:

$ pip3 --version
pip 23.0.1 from /opt/homebrew/lib/python3.11/site-packages/pip (python 3.11)
$ pip3.10 --version
pip 23.0.1 from /opt/homebrew/lib/python3.10/site-packages/pip (python 3.10)

I think the install package should check for Python version and run 3.10, because 3.11 is now installed by default with homebrew.

pojntfx commented 1 year ago

Can confirm this on Fedora 37:

$ uname -a
Linux felicitas-xps13plus 6.1.14-200.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Feb 26 00:13:26 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
$ pip --version
pip 22.2.2 from /usr/lib/python3.11/site-packages/pip (python 3.11)
sieste commented 1 year ago

"i had Python 3.11 set up as default python3 in path, and since there is no wheel for torch for 3.11 yet, the script failed. With 3.10 it worked flawlessly." from this HN comment

cocktailpeanut commented 1 year ago

Thanks for the feedback guys! I actually just returned to my desk and taking a look at all the feedback just now, and also wrote a comment on HN https://news.ycombinator.com/item?id=35129424

Basically, thinking about adding a flag system just to solve the immediate problems. Maybe we can figure out how to automatically detect the binaries, but I think the flag system is a workable solution.

Example:

npx dalai llama 7B 13B --python python3.11 --pip pip23.01

Then instead of the default python dalai will use the specified custom binaries. These can be an absolute path to the actual binary on the system as well.

What do you think?

jooray commented 1 year ago

I think it's perfect

marcuswestin commented 1 year ago

@cocktailpeanut Another approach that might be simpler to add to the current codebase is to make it an environment variable (E.g PYTHON_VERSION=3.11 and PIP_VERSION=23.01; or PYTHON_CMD=python3.11 and PIP_CMD=pip23.01). The default values could be the current ones, or something more appropriate.

marcuswestin commented 1 year ago

Also, #12 has a possibly even better solution which uses virtualenv instead