Closed russellballestrini closed 1 week ago
dleemiller
commented
1 month ago Thanks for testing local compilation. My first instinct is that this has to do with numpy version compatibility. Can you let me know what library version you're using and I'll see if I can reproduce it?
If the issue is only occurring when you truncate the dimension, (for now) you can also download the native 64-dim model like this:
wl = WordLlama.load(dim=64)I'd still like to get to the bottom of the issue though.
chapmanjacobd
commented
1 month ago @russellballestrini try the latest release!
I'm on Fedora 40 with python3.12 and wordllama-0.2.6.post2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl with numpy 2.1.1 works fine for me. Fedora 38 is a tiny bit older but I would be surprised if it didn't work for you as well
russellballestrini
commented
1 month ago (env) (base) [fox@blanka battleship-solvers]$ python3 -m venv env2
(env) (base) [fox@blanka battleship-solvers]$ . env2/bin/activate
(env2) (base) [fox@blanka battleship-solvers]$ python word.py
Traceback (most recent call last):
File "/home/fox/git/russell.ballestrini.net/content/uploads/2024/battleship-solvers/word.py", line 1, in <module>
from wordllama import WordLlama
ModuleNotFoundError: No module named 'wordllama'
(env2) (base) [fox@blanka battleship-solvers]$ pip install wordllama
Collecting wordllama
Downloading wordllama-0.2.6.post2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.7/17.7 MB 21.5 MB/s eta 0:00:00
Collecting numpy
Using cached numpy-2.1.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.3 MB)
Collecting safetensors
Using cached safetensors-0.4.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (435 kB)
Collecting tokenizers
Using cached tokenizers-0.20.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB)
Collecting toml
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting pydantic>=2
Using cached pydantic-2.9.1-py3-none-any.whl (434 kB)
Collecting requests
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Collecting annotated-types>=0.6.0
Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB)
Collecting pydantic-core==2.23.3
Using cached pydantic_core-2.23.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
Collecting typing-extensions>=4.6.1
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Collecting charset-normalizer<4,>=2
Using cached charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Collecting idna<4,>=2.5
Downloading idna-3.10-py3-none-any.whl (70 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.4/70.4 kB 9.0 MB/s eta 0:00:00
Collecting urllib3<3,>=1.21.1
Using cached urllib3-2.2.3-py3-none-any.whl (126 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2024.8.30-py3-none-any.whl (167 kB)
Collecting huggingface-hub<1.0,>=0.16.4
Using cached huggingface_hub-0.24.7-py3-none-any.whl (417 kB)
Collecting filelock
Using cached filelock-3.16.0-py3-none-any.whl (16 kB)
Collecting fsspec>=2023.5.0
Using cached fsspec-2024.9.0-py3-none-any.whl (179 kB)
Collecting packaging>=20.9
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting pyyaml>=5.1
Using cached PyYAML-6.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (762 kB)
Collecting tqdm>=4.42.1
Using cached tqdm-4.66.5-py3-none-any.whl (78 kB)
Installing collected packages: urllib3, typing-extensions, tqdm, toml, safetensors, pyyaml, packaging, numpy, idna, fsspec, filelock, charset-normalizer, certifi, annotated-types, requests, pydantic-core, pydantic, huggingface-hub, tokenizers, wordllama
Successfully installed annotated-types-0.7.0 certifi-2024.8.30 charset-normalizer-3.3.2 filelock-3.16.0 fsspec-2024.9.0 huggingface-hub-0.24.7 idna-3.10 numpy-2.1.1 packaging-24.1 pydantic-2.9.1 pydantic-core-2.23.3 pyyaml-6.0.2 requests-2.32.3 safetensors-0.4.5 tokenizers-0.20.0 toml-0.10.2 tqdm-4.66.5 typing-extensions-4.12.2 urllib3-2.2.3 wordllama-0.2.6.post2
[notice] A new release of pip available: 22.3.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
(env2) (base) [fox@blanka battleship-solvers]$ python word.py
Illegal instruction (core dumped)If I have time, I'll setup an environment and try to reproduce it tonight.
Looks like there are some similar issues previously reported for numpy: https://github.com/numpy/numpy/issues/18131 https://github.com/numpy/numpy/issues/24028
Are you using ARM platform by chance?
russellballestrini
commented
1 month ago Thinkpad T430s (intel i5 dual core, 4 hyperthreads)
dleemiller
commented
1 week ago I think I've got this sorted out in the newest version.
I added a numpy>=2 requirement so that I could change the backend for hamming distance to use this newer bitwise operation in numpy: https://numpy.org/devdocs/reference/generated/numpy.bitwise_count.html
It seems there are some CPUs that don't support popcount instructions. It is simpler to determine at compile time if the hardware is supporting it, that's why it worked when you built it locally. But since we're already using numpy, it makes the most sense to leverage their implementation which is fast and handles all of that complexity already.
Let me know if it doesn't work and I'll think about it some more. I don't think I have anything to test it on, but I'm reasonably certain this is the issue.
russellballestrini
commented
1 week ago @dleemiller thank man for following up. I actually upgraded to fedora 40 from 38 -> 39. I bet the numpy implementation is solid and I will test again when I grok cool ways to implement WordLlama, thanks again for the work.
word.py
I'm not sure if this will automatically fetch the model or if I'm doing something wrong but I don't get any errors besides the core dump.
When I compile myself instead of using pypi it works.