search

voidism / Lookback-Lens

Code for the EMNLP 2024 paper "Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps"
https://arxiv.org/abs/2407.07071
114 stars 7 forks source link

ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects #3

Open nikilpatel94 opened 4 months ago

nikilpatel94 commented 4 months ago

While running the demo, there was a problem of torch 2.0.0 version, but I see there is an issue raised already for this. I removed the version setting against torch in requirements.txt and let it install the latest 2.4.0, but got this error.

Collecting torch (from -r requirements.txt (line 1)) Downloading torch-2.4.0-cp312-none-macosx_11_0_arm64.whl.metadata (26 kB) Collecting numpy==1.26.2 (from -r requirements.txt (line 2)) Downloading numpy-1.26.2-cp312-cp312-macosx_11_0_arm64.whl.metadata (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 4.5 MB/s eta 0:00:00 Requirement already satisfied: scipy in [/Users/iads/.pyenv/versions/3.12.4/lib/python3.12/site-packages](https://file+.vscode-resource.vscode-cdn.net/Users/iads/.pyenv/versions/3.12.4/lib/python3.12/site-packages) (from -r requirements.txt (line 3)) (1.14.0) Collecting tiktoken (from -r requirements.txt (line 4)) Downloading tiktoken-0.7.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (6.6 kB) Collecting openai (from -r requirements.txt (line 5)) Downloading openai-1.37.0-py3-none-any.whl.metadata (22 kB) Collecting editdistance (from -r requirements.txt (line 6)) Downloading editdistance-0.8.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (3.9 kB) Requirement already satisfied: scikit-learn in [/Users/iads/.pyenv/versions/3.12.4/lib/python3.12/site-packages](https://file+.vscode-resource.vscode-cdn.net/Users/iads/.pyenv/versions/3.12.4/lib/python3.12/site-packages) (from -r requirements.txt (line 7)) (1.5.1) Collecting accelerate (from -r requirements.txt (line 8)) Downloading accelerate-0.33.0-py3-none-any.whl.metadata (18 kB) Collecting filelock (from torch->-r requirements.txt (line 1)) Using cached filelock-3.15.4-py3-none-any.whl.metadata (2.9 kB) Collecting typing-extensions>=4.8.0 (from torch->-r requirements.txt (line 1)) Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB) Collecting sympy (from torch->-r requirements.txt (line 1)) Downloading sympy-1.13.1-py3-none-any.whl.metadata (12 kB) Collecting networkx (from torch->-r requirements.txt (line 1)) Downloading networkx-3.3-py3-none-any.whl.metadata (5.1 kB) Collecting jinja2 (from torch->-r requirements.txt (line 1)) Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB) ... ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

Any idea how to fix this?

voidism commented 2 months ago

Hi @nikilpatel94

Can you use python 3.9.5 and cuda toolkit 11.7? I believe you will be able to install torch 2.0.0 in this setting. Thanks!