search

pytorch / torchchat

Run PyTorch LLMs locally on servers, desktop and mobile
BSD 3-Clause "New" or "Revised" License
3.11k stars 193 forks source link

ET Runner build fails with buck2 "Failed to stat `executorch/buck-out/v2`" error #1107

Closed byjlw closed 1 week ago

byjlw commented 1 week ago

🐛 Describe the bug

The following command errors out torchchat/utils/scripts/build_native.sh et

+ '[' 1 -eq 0 ']'
+ ((  1  ))
+ case "$1" in
+ echo 'Building et native runner...'
Building et native runner...
+ TARGET=et
+ shift
+ ((  0  ))
+ '[' -z /Users/jessewhite/Documents/source/torchchat ']'
+ source /Users/jessewhite/Documents/source/torchchat/torchchat/utils/scripts/install_utils.sh
++ set -ex pipefail
++ COMMON_CMAKE_ARGS='    -DCMAKE_BUILD_TYPE=Release     -DEXECUTORCH_ENABLE_LOGGING=ON     -DEXECUTORCH_LOG_LEVEL=Info     -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON     -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON     -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON     -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON     -DEXECUTORCH_BUILD_XNNPACK=ON'
+ '[' -z '' ']'
+ ET_BUILD_DIR=et-build
+ pushd /Users/jessewhite/Documents/source/torchchat
~/Documents/source/torchchat ~/Documents/source/torchchat
+ git submodule update --init
+ git submodule sync
Synchronizing submodule url for 'tokenizer/third-party/abseil-cpp'
Synchronizing submodule url for 'tokenizer/third-party/re2'
Synchronizing submodule url for 'tokenizer/third-party/sentencepiece'
+ [[ et == \e\t ]]
+ find_cmake_prefix_path
++ python3 -c 'from distutils.sysconfig import get_python_lib;print(get_python_lib())'
+ path=/Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages
+ MY_CMAKE_PREFIX_PATH=/Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages
+ install_pip_dependencies
+ echo 'Intalling common pip packages'
Intalling common pip packages
+ pip3 install wheel 'cmake>=3.19' ninja zstd
Requirement already satisfied: wheel in ./.venv/lib/python3.11/site-packages (0.44.0)
Requirement already satisfied: cmake>=3.19 in ./.venv/lib/python3.11/site-packages (3.30.2)
Requirement already satisfied: ninja in ./.venv/lib/python3.11/site-packages (1.11.1.1)
Requirement already satisfied: zstd in ./.venv/lib/python3.11/site-packages (1.5.5.1)

[notice] A new release of pip is available: 24.0 -> 24.2
[notice] To update, run: pip install --upgrade pip
+ pushd /Users/jessewhite/Documents/source/torchchat
~/Documents/source/torchchat ~/Documents/source/torchchat ~/Documents/source/torchchat
+ pip3 install -r install/requirements.txt
Ignoring tomli: markers 'python_version < "3.11"' don't match your environment
Requirement already satisfied: huggingface_hub in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 4)) (0.24.6)
Requirement already satisfied: gguf in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 7)) (0.10.0)
Requirement already satisfied: tiktoken in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 10)) (0.7.0)
Requirement already satisfied: snakeviz in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 13)) (2.2.0)
Requirement already satisfied: sentencepiece in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 14)) (0.2.0)
Requirement already satisfied: numpy<2.0 in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 15)) (1.26.4)
Requirement already satisfied: lm-eval==0.4.2 in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 17)) (0.4.2)
Requirement already satisfied: blobfile in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 18)) (3.0.0)
Requirement already satisfied: openai in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 20)) (1.43.0)
Requirement already satisfied: wheel in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 23)) (0.44.0)
Requirement already satisfied: cmake>=3.24 in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 24)) (3.30.2)
Requirement already satisfied: ninja in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 25)) (1.11.1.1)
Requirement already satisfied: zstd in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 26)) (1.5.5.1)
Requirement already satisfied: streamlit in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 29)) (1.38.0)
Requirement already satisfied: flask in ./.venv/lib/python3.11/site-packages (from -r install/requirements.txt (line 32)) (3.0.3)
Requirement already satisfied: accelerate>=0.21.0 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.34.0)
Requirement already satisfied: evaluate in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.4.2)
Requirement already satisfied: datasets>=2.16.0 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.21.0)
Requirement already satisfied: jsonlines in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (4.0.0)
Requirement already satisfied: numexpr in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.10.1)
Requirement already satisfied: peft>=0.2.0 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.12.0)
Requirement already satisfied: pybind11>=2.6.2 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.13.5)
Requirement already satisfied: pytablewriter in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.2.0)
Requirement already satisfied: rouge-score>=0.0.4 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.1.2)
Requirement already satisfied: sacrebleu>=1.5.0 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.4.3)
Requirement already satisfied: scikit-learn>=0.24.1 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.5.1)
Requirement already satisfied: sqlitedict in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.1.0)
Requirement already satisfied: torch>=1.8 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.5.0.dev20240716)
Requirement already satisfied: tqdm-multiprocess in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.0.11)
Requirement already satisfied: transformers>=4.1 in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (4.42.4)
Requirement already satisfied: zstandard in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.23.0)
Requirement already satisfied: dill in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.3.8)
Requirement already satisfied: word2number in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.1)
Requirement already satisfied: more-itertools in ./.venv/lib/python3.11/site-packages (from lm-eval==0.4.2->-r install/requirements.txt (line 17)) (10.4.0)
Requirement already satisfied: filelock in ./.venv/lib/python3.11/site-packages (from huggingface_hub->-r install/requirements.txt (line 4)) (3.15.4)
Requirement already satisfied: fsspec>=2023.5.0 in ./.venv/lib/python3.11/site-packages (from huggingface_hub->-r install/requirements.txt (line 4)) (2024.6.1)
Requirement already satisfied: packaging>=20.9 in ./.venv/lib/python3.11/site-packages (from huggingface_hub->-r install/requirements.txt (line 4)) (24.1)
Requirement already satisfied: pyyaml>=5.1 in ./.venv/lib/python3.11/site-packages (from huggingface_hub->-r install/requirements.txt (line 4)) (6.0.2)
Requirement already satisfied: requests in ./.venv/lib/python3.11/site-packages (from huggingface_hub->-r install/requirements.txt (line 4)) (2.32.3)
Requirement already satisfied: tqdm>=4.42.1 in ./.venv/lib/python3.11/site-packages (from huggingface_hub->-r install/requirements.txt (line 4)) (4.66.5)
Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.venv/lib/python3.11/site-packages (from huggingface_hub->-r install/requirements.txt (line 4)) (4.12.2)
Requirement already satisfied: regex>=2022.1.18 in ./.venv/lib/python3.11/site-packages (from tiktoken->-r install/requirements.txt (line 10)) (2024.7.24)
Requirement already satisfied: tornado>=2.0 in ./.venv/lib/python3.11/site-packages (from snakeviz->-r install/requirements.txt (line 13)) (6.4.1)
Requirement already satisfied: pycryptodomex>=3.8 in ./.venv/lib/python3.11/site-packages (from blobfile->-r install/requirements.txt (line 18)) (3.20.0)
Requirement already satisfied: urllib3<3,>=1.25.3 in ./.venv/lib/python3.11/site-packages (from blobfile->-r install/requirements.txt (line 18)) (2.2.2)
Requirement already satisfied: lxml>=4.9 in ./.venv/lib/python3.11/site-packages (from blobfile->-r install/requirements.txt (line 18)) (5.3.0)
Requirement already satisfied: anyio<5,>=3.5.0 in ./.venv/lib/python3.11/site-packages (from openai->-r install/requirements.txt (line 20)) (4.4.0)
Requirement already satisfied: distro<2,>=1.7.0 in ./.venv/lib/python3.11/site-packages (from openai->-r install/requirements.txt (line 20)) (1.9.0)
Requirement already satisfied: httpx<1,>=0.23.0 in ./.venv/lib/python3.11/site-packages (from openai->-r install/requirements.txt (line 20)) (0.27.2)
Requirement already satisfied: jiter<1,>=0.4.0 in ./.venv/lib/python3.11/site-packages (from openai->-r install/requirements.txt (line 20)) (0.5.0)
Requirement already satisfied: pydantic<3,>=1.9.0 in ./.venv/lib/python3.11/site-packages (from openai->-r install/requirements.txt (line 20)) (2.8.2)
Requirement already satisfied: sniffio in ./.venv/lib/python3.11/site-packages (from openai->-r install/requirements.txt (line 20)) (1.3.1)
Requirement already satisfied: altair<6,>=4.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (5.4.1)
Requirement already satisfied: blinker<2,>=1.0.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (1.8.2)
Requirement already satisfied: cachetools<6,>=4.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (5.5.0)
Requirement already satisfied: click<9,>=7.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (8.1.7)
Requirement already satisfied: pandas<3,>=1.3.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (2.2.2)
Requirement already satisfied: pillow<11,>=7.1.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (10.4.0)
Requirement already satisfied: protobuf<6,>=3.20 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (5.28.0)
Requirement already satisfied: pyarrow>=7.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (17.0.0)
Requirement already satisfied: rich<14,>=10.14.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (13.8.0)
Requirement already satisfied: tenacity<9,>=8.1.0 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (8.5.0)
Requirement already satisfied: toml<2,>=0.10.1 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (0.10.2)
Requirement already satisfied: gitpython!=3.1.19,<4,>=3.0.7 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (3.1.43)
Requirement already satisfied: pydeck<1,>=0.8.0b4 in ./.venv/lib/python3.11/site-packages (from streamlit->-r install/requirements.txt (line 29)) (0.9.1)
Requirement already satisfied: Werkzeug>=3.0.0 in ./.venv/lib/python3.11/site-packages (from flask->-r install/requirements.txt (line 32)) (3.0.4)
Requirement already satisfied: Jinja2>=3.1.2 in ./.venv/lib/python3.11/site-packages (from flask->-r install/requirements.txt (line 32)) (3.1.4)
Requirement already satisfied: itsdangerous>=2.1.2 in ./.venv/lib/python3.11/site-packages (from flask->-r install/requirements.txt (line 32)) (2.2.0)
Requirement already satisfied: psutil in ./.venv/lib/python3.11/site-packages (from accelerate>=0.21.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (6.0.0)
Requirement already satisfied: safetensors>=0.4.3 in ./.venv/lib/python3.11/site-packages (from accelerate>=0.21.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.4.4)
Requirement already satisfied: jsonschema>=3.0 in ./.venv/lib/python3.11/site-packages (from altair<6,>=4.0->streamlit->-r install/requirements.txt (line 29)) (4.23.0)
Requirement already satisfied: narwhals>=1.5.2 in ./.venv/lib/python3.11/site-packages (from altair<6,>=4.0->streamlit->-r install/requirements.txt (line 29)) (1.6.2)
Requirement already satisfied: idna>=2.8 in ./.venv/lib/python3.11/site-packages (from anyio<5,>=3.5.0->openai->-r install/requirements.txt (line 20)) (3.8)
Requirement already satisfied: xxhash in ./.venv/lib/python3.11/site-packages (from datasets>=2.16.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (3.5.0)
Requirement already satisfied: multiprocess in ./.venv/lib/python3.11/site-packages (from datasets>=2.16.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.70.16)
Requirement already satisfied: aiohttp in ./.venv/lib/python3.11/site-packages (from datasets>=2.16.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (3.10.5)
Requirement already satisfied: gitdb<5,>=4.0.1 in ./.venv/lib/python3.11/site-packages (from gitpython!=3.1.19,<4,>=3.0.7->streamlit->-r install/requirements.txt (line 29)) (4.0.11)
Requirement already satisfied: certifi in ./.venv/lib/python3.11/site-packages (from httpx<1,>=0.23.0->openai->-r install/requirements.txt (line 20)) (2024.8.30)
Requirement already satisfied: httpcore==1.* in ./.venv/lib/python3.11/site-packages (from httpx<1,>=0.23.0->openai->-r install/requirements.txt (line 20)) (1.0.5)
Requirement already satisfied: h11<0.15,>=0.13 in ./.venv/lib/python3.11/site-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai->-r install/requirements.txt (line 20)) (0.14.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./.venv/lib/python3.11/site-packages (from Jinja2>=3.1.2->flask->-r install/requirements.txt (line 32)) (2.1.5)
Requirement already satisfied: python-dateutil>=2.8.2 in ./.venv/lib/python3.11/site-packages (from pandas<3,>=1.3.0->streamlit->-r install/requirements.txt (line 29)) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in ./.venv/lib/python3.11/site-packages (from pandas<3,>=1.3.0->streamlit->-r install/requirements.txt (line 29)) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in ./.venv/lib/python3.11/site-packages (from pandas<3,>=1.3.0->streamlit->-r install/requirements.txt (line 29)) (2024.1)
Requirement already satisfied: annotated-types>=0.4.0 in ./.venv/lib/python3.11/site-packages (from pydantic<3,>=1.9.0->openai->-r install/requirements.txt (line 20)) (0.7.0)
Requirement already satisfied: pydantic-core==2.20.1 in ./.venv/lib/python3.11/site-packages (from pydantic<3,>=1.9.0->openai->-r install/requirements.txt (line 20)) (2.20.1)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.venv/lib/python3.11/site-packages (from requests->huggingface_hub->-r install/requirements.txt (line 4)) (3.3.2)
Requirement already satisfied: markdown-it-py>=2.2.0 in ./.venv/lib/python3.11/site-packages (from rich<14,>=10.14.0->streamlit->-r install/requirements.txt (line 29)) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in ./.venv/lib/python3.11/site-packages (from rich<14,>=10.14.0->streamlit->-r install/requirements.txt (line 29)) (2.18.0)
Requirement already satisfied: absl-py in ./.venv/lib/python3.11/site-packages (from rouge-score>=0.0.4->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.1.0)
Requirement already satisfied: nltk in ./.venv/lib/python3.11/site-packages (from rouge-score>=0.0.4->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (3.9.1)
Requirement already satisfied: six>=1.14.0 in ./.venv/lib/python3.11/site-packages (from rouge-score>=0.0.4->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.16.0)
Requirement already satisfied: portalocker in ./.venv/lib/python3.11/site-packages (from sacrebleu>=1.5.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.10.1)
Requirement already satisfied: tabulate>=0.8.9 in ./.venv/lib/python3.11/site-packages (from sacrebleu>=1.5.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.9.0)
Requirement already satisfied: colorama in ./.venv/lib/python3.11/site-packages (from sacrebleu>=1.5.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.4.6)
Requirement already satisfied: scipy>=1.6.0 in ./.venv/lib/python3.11/site-packages (from scikit-learn>=0.24.1->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.14.1)
Requirement already satisfied: joblib>=1.2.0 in ./.venv/lib/python3.11/site-packages (from scikit-learn>=0.24.1->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.4.2)
Requirement already satisfied: threadpoolctl>=3.1.0 in ./.venv/lib/python3.11/site-packages (from scikit-learn>=0.24.1->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (3.5.0)
Requirement already satisfied: sympy in ./.venv/lib/python3.11/site-packages (from torch>=1.8->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.13.1)
Requirement already satisfied: networkx in ./.venv/lib/python3.11/site-packages (from torch>=1.8->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (3.3)
Requirement already satisfied: tokenizers<0.20,>=0.19 in ./.venv/lib/python3.11/site-packages (from transformers>=4.1->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.19.1)
Requirement already satisfied: attrs>=19.2.0 in ./.venv/lib/python3.11/site-packages (from jsonlines->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (24.2.0)
Requirement already satisfied: setuptools>=38.3.0 in ./.venv/lib/python3.11/site-packages (from pytablewriter->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (65.5.0)
Requirement already satisfied: DataProperty<2,>=1.0.1 in ./.venv/lib/python3.11/site-packages (from pytablewriter->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.0.1)
Requirement already satisfied: mbstrdecoder<2,>=1.0.0 in ./.venv/lib/python3.11/site-packages (from pytablewriter->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.1.3)
Requirement already satisfied: pathvalidate<4,>=2.3.0 in ./.venv/lib/python3.11/site-packages (from pytablewriter->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (3.2.1)
Requirement already satisfied: tabledata<2,>=1.3.1 in ./.venv/lib/python3.11/site-packages (from pytablewriter->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.3.3)
Requirement already satisfied: tcolorpy<1,>=0.0.5 in ./.venv/lib/python3.11/site-packages (from pytablewriter->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (0.1.6)
Requirement already satisfied: typepy<2,>=1.3.2 in ./.venv/lib/python3.11/site-packages (from typepy[datetime]<2,>=1.3.2->pytablewriter->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.3.2)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in ./.venv/lib/python3.11/site-packages (from aiohttp->datasets>=2.16.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (2.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in ./.venv/lib/python3.11/site-packages (from aiohttp->datasets>=2.16.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.3.1)
Requirement already satisfied: frozenlist>=1.1.1 in ./.venv/lib/python3.11/site-packages (from aiohttp->datasets>=2.16.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in ./.venv/lib/python3.11/site-packages (from aiohttp->datasets>=2.16.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in ./.venv/lib/python3.11/site-packages (from aiohttp->datasets>=2.16.0->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.9.8)
Requirement already satisfied: smmap<6,>=3.0.1 in ./.venv/lib/python3.11/site-packages (from gitdb<5,>=4.0.1->gitpython!=3.1.19,<4,>=3.0.7->streamlit->-r install/requirements.txt (line 29)) (5.0.1)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in ./.venv/lib/python3.11/site-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit->-r install/requirements.txt (line 29)) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in ./.venv/lib/python3.11/site-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit->-r install/requirements.txt (line 29)) (0.35.1)
Requirement already satisfied: rpds-py>=0.7.1 in ./.venv/lib/python3.11/site-packages (from jsonschema>=3.0->altair<6,>=4.0->streamlit->-r install/requirements.txt (line 29)) (0.20.0)
Requirement already satisfied: mdurl~=0.1 in ./.venv/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich<14,>=10.14.0->streamlit->-r install/requirements.txt (line 29)) (0.1.2)
Requirement already satisfied: chardet<6,>=3.0.4 in ./.venv/lib/python3.11/site-packages (from mbstrdecoder<2,>=1.0.0->pytablewriter->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (5.2.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./.venv/lib/python3.11/site-packages (from sympy->torch>=1.8->lm-eval==0.4.2->-r install/requirements.txt (line 17)) (1.3.0)

[notice] A new release of pip is available: 24.0 -> 24.2
[notice] To update, run: pip install --upgrade pip
+ popd
~/Documents/source/torchchat ~/Documents/source/torchchat
+ clone_executorch
+ echo 'Cloning executorch to /Users/jessewhite/Documents/source/torchchat/et-build/src'
Cloning executorch to /Users/jessewhite/Documents/source/torchchat/et-build/src
+ rm -rf /Users/jessewhite/Documents/source/torchchat/et-build
+ mkdir -p /Users/jessewhite/Documents/source/torchchat/et-build/src
+ pushd /Users/jessewhite/Documents/source/torchchat/et-build/src
~/Documents/source/torchchat/et-build/src ~/Documents/source/torchchat ~/Documents/source/torchchat
+ git clone https://github.com/pytorch/executorch.git
Cloning into 'executorch'...
remote: Enumerating objects: 115556, done.
remote: Counting objects: 100% (2960/2960), done.
remote: Compressing objects: 100% (1288/1288), done.
remote: Total 115556 (delta 1935), reused 2510 (delta 1603), pack-reused 112596 (from 1)
Receiving objects: 100% (115556/115556), 85.81 MiB | 7.51 MiB/s, done.
Resolving deltas: 100% (89385/89385), done.
+ cd executorch
++ cat /Users/jessewhite/Documents/source/torchchat/install/.pins/et-pin.txt
+ git checkout 91298923a0076c1b41059efb6dad2876426e4b03
Note: switching to '91298923a0076c1b41059efb6dad2876426e4b03'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 91298923a immutable accessors in graph signature (#4433)
+ echo 'Install executorch: submodule update'
Install executorch: submodule update
+ git submodule sync
+ git submodule update --init
Submodule 'backends/arm/third-party/ethos-u-core-driver' (https://review.mlplatform.org/ml/ethos-u/ethos-u-core-driver) registered for path 'backends/arm/third-party/ethos-u-core-driver'
Submodule 'backends/arm/third-party/serialization_lib' (https://review.mlplatform.org/tosa/serialization_lib) registered for path 'backends/arm/third-party/serialization_lib'
Submodule 'backends/vulkan/third-party/Vulkan-Headers' (https://github.com/KhronosGroup/Vulkan-Headers) registered for path 'backends/vulkan/third-party/Vulkan-Headers'
Submodule 'backends/vulkan/third-party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'backends/vulkan/third-party/VulkanMemoryAllocator'
Submodule 'backends/vulkan/third-party/volk' (https://github.com/zeux/volk) registered for path 'backends/vulkan/third-party/volk'
Submodule 'backends/xnnpack/third-party/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'backends/xnnpack/third-party/FP16'
Submodule 'backends/xnnpack/third-party/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'backends/xnnpack/third-party/FXdiv'
Submodule 'backends/xnnpack/third-party/XNNPACK' (https://github.com/digantdesai/XNNPACK.git) registered for path 'backends/xnnpack/third-party/XNNPACK'
Submodule 'backends/xnnpack/third-party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'backends/xnnpack/third-party/cpuinfo'
Submodule 'backends/xnnpack/third-party/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'backends/xnnpack/third-party/pthreadpool'
Submodule 'examples/third-party/LLaVA' (https://github.com/haotian-liu/LLaVA.git) registered for path 'examples/third-party/LLaVA'
Submodule 'examples/third-party/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'examples/third-party/fbjni'
Submodule 'extension/llm/third-party/abseil-cpp' (https://github.com/abseil/abseil-cpp.git) registered for path 'extension/llm/third-party/abseil-cpp'
Submodule 'extension/llm/third-party/re2' (https://github.com/google/re2.git) registered for path 'extension/llm/third-party/re2'
Submodule 'extension/llm/third-party/sentencepiece' (https://github.com/google/sentencepiece.git) registered for path 'extension/llm/third-party/sentencepiece'
Submodule 'kernels/optimized/third-party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'kernels/optimized/third-party/eigen'
Submodule 'third-party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third-party/flatbuffers'
Submodule 'third-party/flatcc' (https://github.com/dvidelabs/flatcc.git) registered for path 'third-party/flatcc'
Submodule 'third-party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third-party/gflags'
Submodule 'third-party/googletest' (https://github.com/google/googletest.git) registered for path 'third-party/googletest'
Submodule 'third-party/ios-cmake' (https://github.com/leetal/ios-cmake) registered for path 'third-party/ios-cmake'
Submodule 'third-party/prelude' (https://github.com/facebook/buck2-prelude.git) registered for path 'third-party/prelude'
Submodule 'third-party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third-party/pybind11'
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/arm/third-party/ethos-u-core-driver'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/arm/third-party/serialization_lib'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/vulkan/third-party/Vulkan-Headers'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/vulkan/third-party/VulkanMemoryAllocator'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/vulkan/third-party/volk'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/xnnpack/third-party/FP16'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/xnnpack/third-party/FXdiv'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/xnnpack/third-party/XNNPACK'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/xnnpack/third-party/cpuinfo'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/backends/xnnpack/third-party/pthreadpool'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/examples/third-party/LLaVA'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/examples/third-party/fbjni'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/extension/llm/third-party/abseil-cpp'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/extension/llm/third-party/re2'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/extension/llm/third-party/sentencepiece'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/kernels/optimized/third-party/eigen'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/third-party/flatbuffers'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/third-party/flatcc'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/third-party/gflags'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/third-party/googletest'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/third-party/ios-cmake'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/third-party/prelude'...
Cloning into '/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/third-party/pybind11'...
Submodule path 'backends/arm/third-party/ethos-u-core-driver': checked out '90f9df900acdc0718ecd2dfdc53780664758dec5'
Submodule path 'backends/arm/third-party/serialization_lib': checked out '187af0d41fe75d08d2a7ec84c1b4d24b9b641ed2'
Submodule path 'backends/vulkan/third-party/Vulkan-Headers': checked out '0c5928795a66e93f65e5e68a36d8daa79a209dc2'
Submodule path 'backends/vulkan/third-party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191'
Submodule path 'backends/vulkan/third-party/volk': checked out 'b3bc21e584f97400b6884cb2a541a56c6a5ddba3'
Submodule path 'backends/xnnpack/third-party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3'
Submodule path 'backends/xnnpack/third-party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1'
Submodule path 'backends/xnnpack/third-party/XNNPACK': checked out '1d139a3b4b7155889c88c31f370a82c48e7ca89c'
Submodule path 'backends/xnnpack/third-party/cpuinfo': checked out 'd6860c477c99f1fce9e28eb206891af3c0e1a1d7'
Submodule path 'backends/xnnpack/third-party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8'
Submodule path 'examples/third-party/LLaVA': checked out '7440ec9ee37b0374c6b5548818e89878e38f3353'
Submodule path 'examples/third-party/fbjni': checked out '52a14f0daa889a20d8984798b8d96eb03cebd334'
Submodule path 'extension/llm/third-party/abseil-cpp': checked out 'eb852207758a773965301d0ae717e4235fc5301a'
Submodule path 'extension/llm/third-party/re2': checked out '6dcd83d60f7944926bfd308cc13979fc53dd69ca'
Submodule path 'extension/llm/third-party/sentencepiece': checked out '6225e08edb2577757163b3f5dbba4c0b670ef445'
Submodule path 'kernels/optimized/third-party/eigen': checked out 'a39ade4ccf99df845ec85c580fbbb324f71952fa'
Submodule path 'third-party/flatbuffers': checked out '595bf0007ab1929570c7671f091313c8fc20644e'
Submodule path 'third-party/flatcc': checked out '896db54787e8b730a6be482c69324751f3f5f117'
Submodule path 'third-party/gflags': checked out 'a738fdf9338412f83ab3f26f31ac11ed3f3ec4bd'
Submodule path 'third-party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
Submodule path 'third-party/ios-cmake': checked out '06465b27698424cf4a04a5ca4904d50a3c966c45'
Submodule path 'third-party/prelude': checked out '4e9e6d50b8b461564a7e351ff60b87fe59d7e53b'
Submodule path 'third-party/pybind11': checked out '8c7b8dd0ae74b36b7d42f77b0dd4096ebb7f4ab1'
+ popd
~/Documents/source/torchchat ~/Documents/source/torchchat
+ install_executorch_libs false
+ export 'CMAKE_ARGS=        -DCMAKE_BUILD_TYPE=Release     -DEXECUTORCH_ENABLE_LOGGING=ON     -DEXECUTORCH_LOG_LEVEL=Info     -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON     -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON     -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON     -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON     -DEXECUTORCH_BUILD_XNNPACK=ON     -DCMAKE_PREFIX_PATH=/Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages     -DCMAKE_INSTALL_PREFIX=/Users/jessewhite/Documents/source/torchchat/et-build/install'
+ CMAKE_ARGS='        -DCMAKE_BUILD_TYPE=Release     -DEXECUTORCH_ENABLE_LOGGING=ON     -DEXECUTORCH_LOG_LEVEL=Info     -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON     -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON     -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON     -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON     -DEXECUTORCH_BUILD_XNNPACK=ON     -DCMAKE_PREFIX_PATH=/Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages     -DCMAKE_INSTALL_PREFIX=/Users/jessewhite/Documents/source/torchchat/et-build/install'
+ export 'CMAKE_BUILD_ARGS=--target install'
+ CMAKE_BUILD_ARGS='--target install'
+ install_executorch_python_libs false
+ '[' '!' -d /Users/jessewhite/Documents/source/torchchat/et-build ']'
+ pushd /Users/jessewhite/Documents/source/torchchat/et-build/src
~/Documents/source/torchchat/et-build/src ~/Documents/source/torchchat ~/Documents/source/torchchat
+ cd executorch
+ echo 'Building and installing python libraries'
Building and installing python libraries
+ '[' '' = false ']'
+ echo 'Installing pybind'
Installing pybind
+ bash ./install_requirements.sh --pybind xnnpack
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/nightly/cpu
Requirement already satisfied: torch==2.5.0.dev20240716 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (2.5.0.dev20240716)
Requirement already satisfied: torchvision==0.20.0.dev20240716 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (0.20.0.dev20240716)
Requirement already satisfied: cmake in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (3.30.2)
Requirement already satisfied: pip>=23 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (24.0)
Requirement already satisfied: pyyaml in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (6.0.2)
Requirement already satisfied: setuptools>=63 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (65.5.0)
Requirement already satisfied: tomli in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (2.0.1)
Requirement already satisfied: wheel in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (0.44.0)
Requirement already satisfied: zstd in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (1.5.5.1)
Requirement already satisfied: timm==1.0.7 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (1.0.7)
Requirement already satisfied: torchaudio==2.4.0.dev20240716 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (2.4.0.dev20240716)
Requirement already satisfied: torchsr==1.0.4 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (1.0.4)
Requirement already satisfied: transformers==4.42.4 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (4.42.4)
Requirement already satisfied: filelock in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from torch==2.5.0.dev20240716) (3.15.4)
Requirement already satisfied: typing-extensions>=4.8.0 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from torch==2.5.0.dev20240716) (4.12.2)
Requirement already satisfied: sympy in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from torch==2.5.0.dev20240716) (1.13.1)
Requirement already satisfied: networkx in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from torch==2.5.0.dev20240716) (3.3)
Requirement already satisfied: jinja2 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from torch==2.5.0.dev20240716) (3.1.4)
Requirement already satisfied: fsspec in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from torch==2.5.0.dev20240716) (2024.6.1)
Requirement already satisfied: numpy in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from torchvision==0.20.0.dev20240716) (1.26.4)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from torchvision==0.20.0.dev20240716) (10.4.0)
Requirement already satisfied: huggingface_hub in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from timm==1.0.7) (0.24.6)
Requirement already satisfied: safetensors in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from timm==1.0.7) (0.4.4)
Requirement already satisfied: packaging>=20.0 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from transformers==4.42.4) (24.1)
Requirement already satisfied: regex!=2019.12.17 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from transformers==4.42.4) (2024.7.24)
Requirement already satisfied: requests in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from transformers==4.42.4) (2.32.3)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from transformers==4.42.4) (0.19.1)
Requirement already satisfied: tqdm>=4.27 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from transformers==4.42.4) (4.66.5)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from jinja2->torch==2.5.0.dev20240716) (2.1.5)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from requests->transformers==4.42.4) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from requests->transformers==4.42.4) (3.8)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from requests->transformers==4.42.4) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from requests->transformers==4.42.4) (2024.8.30)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from sympy->torch==2.5.0.dev20240716) (1.3.0)

[notice] A new release of pip is available: 24.0 -> 24.2
[notice] To update, run: pip install --upgrade pip
Using pip 24.0 from /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages/pip (python 3.11)
WARNING: The index url "" seems invalid, please provide a scheme.
Looking in indexes: https://pypi.org/simple, 
Processing /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch
  Running command Preparing metadata (pyproject.toml)
  /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
    warnings.warn(msg, _BetaConfiguration)
  running dist_info
  creating /private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info
  writing /private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info/PKG-INFO
  writing dependency_links to /private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info/dependency_links.txt
  writing entry points to /private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info/entry_points.txt
  writing requirements to /private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info/requires.txt
  writing top-level names to /private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info/top_level.txt
  writing manifest file '/private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info/SOURCES.txt'
  reading manifest file '/private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info/SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file '/private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch.egg-info/SOURCES.txt'
  creating '/private/var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/pip-modern-metadata-kos3pdfu/executorch-0.4.0a0+9129892.dist-info'
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: expecttest in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (0.2.1)
Requirement already satisfied: flatbuffers in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (24.3.25)
Requirement already satisfied: hypothesis in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (6.111.2)
Requirement already satisfied: mpmath==1.3.0 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (1.3.0)
Requirement already satisfied: numpy>=1.25.2 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (1.26.4)
Requirement already satisfied: packaging in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (24.1)
Requirement already satisfied: pandas in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (2.2.2)
Requirement already satisfied: parameterized in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (0.9.0)
Requirement already satisfied: pytest in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (8.3.2)
Requirement already satisfied: pytest-xdist in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (3.6.1)
Requirement already satisfied: pyyaml in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (6.0.2)
Requirement already satisfied: ruamel.yaml in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (0.18.6)
Requirement already satisfied: sympy in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (1.13.1)
Requirement already satisfied: tabulate in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from executorch==0.4.0a0+9129892) (0.9.0)
Requirement already satisfied: attrs>=22.2.0 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from hypothesis->executorch==0.4.0a0+9129892) (24.2.0)
Requirement already satisfied: sortedcontainers<3.0.0,>=2.1.0 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from hypothesis->executorch==0.4.0a0+9129892) (2.4.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from pandas->executorch==0.4.0a0+9129892) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from pandas->executorch==0.4.0a0+9129892) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from pandas->executorch==0.4.0a0+9129892) (2024.1)
Requirement already satisfied: iniconfig in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from pytest->executorch==0.4.0a0+9129892) (2.0.0)
Requirement already satisfied: pluggy<2,>=1.5 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from pytest->executorch==0.4.0a0+9129892) (1.5.0)
Requirement already satisfied: execnet>=2.1 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from pytest-xdist->executorch==0.4.0a0+9129892) (2.1.1)
Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from ruamel.yaml->executorch==0.4.0a0+9129892) (0.2.8)
Requirement already satisfied: six>=1.5 in /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->executorch==0.4.0a0+9129892) (1.16.0)
Building wheels for collected packages: executorch
  Running command Building wheel for executorch (pyproject.toml)
  /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
    warnings.warn(msg, _BetaConfiguration)
  running bdist_wheel
  running build
  command options for 'CustomBuild':
    build_base = pip-out
    build_purelib = pip-out/lib
    build_platlib = pip-out/lib.macosx-10.9-universal2-cpython-311
    build_lib = pip-out/lib.macosx-10.9-universal2-cpython-311
    build_scripts = pip-out/scripts-3.11
    build_temp = pip-out/temp.macosx-10.9-universal2-cpython-311
    plat_name = macosx-10.9-universal2
    compiler = None
    parallel = 15
    debug = None
    force = None
    executable = /Users/jessewhite/Documents/source/torchchat/.venv/bin/python3.11
  creating /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out
  creating /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311
  creating /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out
  deleting /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/CMakeCache.txt
  cmake -S /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch -B /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out -DBUCK2= -DPYTHON_EXECUTABLE=/Users/jessewhite/Documents/source/torchchat/.venv/bin/python3.11 -DCMAKE_PREFIX_PATH=/Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages -DCMAKE_BUILD_TYPE=Release -DEXECUTORCH_ENABLE_LOGGING=ON -DEXECUTORCH_LOG_LEVEL=Info -DCMAKE_OSX_DEPLOYMENT_TARGET=10.15 -DEXECUTORCH_SEPARATE_FLATCC_HOST_PROJECT=OFF -DEXECUTORCH_BUILD_PYBIND=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED_AOT=ON -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON -DEXECUTORCH_BUILD_KERNELS_CUSTOM_AOT=ON -DCMAKE_BUILD_TYPE=Release -DEXECUTORCH_ENABLE_LOGGING=ON -DEXECUTORCH_LOG_LEVEL=Info -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON -DEXECUTORCH_BUILD_XNNPACK=ON -DCMAKE_PREFIX_PATH=/Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages -DCMAKE_INSTALL_PREFIX=/Users/jessewhite/Documents/source/torchchat/et-build/install -DEXECUTORCH_BUILD_XNNPACK=ON
  -- The C compiler identification is AppleClang 15.0.0.15000100
  -- The CXX compiler identification is AppleClang 15.0.0.15000100
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Downloading FXdiv to /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/FXdiv-source (define FXDIV_SOURCE_DIR to avoid it)
  -- Configuring done (0.0s)
  -- Generating done (0.0s)
  -- Build files have been written to: /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/FXdiv-download
  [ 11%] Creating directories for 'fxdiv'
  [ 22%] Performing download step (git clone) for 'fxdiv'
  Cloning into 'FXdiv-source'...
  Already on 'master'
  Your branch is up to date with 'origin/master'.
  [ 33%] Performing update step for 'fxdiv'
  -- Fetching latest from the remote origin
  [ 44%] No patch step for 'fxdiv'
  [ 55%] No configure step for 'fxdiv'
  [ 66%] No build step for 'fxdiv'
  [ 77%] No install step for 'fxdiv'
  [ 88%] No test step for 'fxdiv'
  [100%] Completed 'fxdiv'
  [100%] Built target fxdiv
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
  -- Found Threads: TRUE
  -- Using python executable '/Users/jessewhite/Documents/source/torchchat/.venv/bin/python3.11'
  -- Resolved buck2 as /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/buck2-bin/buck2-99773fe6f7963a72ae5f7b737c02836e.
  -- Killing buck2 daemon
  -- executorch: Generating source lists
  -- executorch: Generating source file list /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/executorch_srcs.cmake
  Error while generating /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/executorch_srcs.cmake. Exit code: 1
  Output:

  Error:
  Traceback (most recent call last):
    File "/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/build/buck_util.py", line 26, in run
      cp: subprocess.CompletedProcess = subprocess.run(
                                        ^^^^^^^^^^^^^^^
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/buck2-bin/buck2-99773fe6f7963a72ae5f7b737c02836e', 'cquery', "inputs(deps('//runtime/executor:program'))"]' returned non-zero exit status 2.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/build/extract_sources.py", line 218, in <module>
      main()
    File "/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/build/extract_sources.py", line 203, in main
      target_to_srcs[name] = sorted(target.get_sources(graph, runner))
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/build/extract_sources.py", line 116, in get_sources
      sources: set[str] = set(runner.run(["cquery", query]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/build/buck_util.py", line 31, in run
      raise RuntimeError(ex.stderr.decode("utf-8")) from ex
  RuntimeError: Command failed:
  Error validating working directory

  Caused by:
      0: Failed to stat `/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/buck-out/v2`
      1: ENOENT: No such file or directory

  CMake Error at build/Utils.cmake:191 (message):
    executorch: source list generation failed
  Call Stack (most recent call first):
    CMakeLists.txt:327 (extract_sources)

  -- Configuring incomplete, errors occurred!
  error: command '/Users/jessewhite/Documents/source/torchchat/.venv/bin/cmake' failed with exit code 1
  error: subprocess-exited-with-error

  × Building wheel for executorch (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /Users/jessewhite/Documents/source/torchchat/.venv/bin/python3.11 /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /var/folders/7k/2n2lk60d5n57_33yh299tt0c0000gn/T/tmpx76eqaoy
  cwd: /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch
  Building wheel for executorch (pyproject.toml) ... error
  ERROR: Failed building wheel for executorch
Failed to build executorch
ERROR: Could not build wheels for executorch, which is required to install pyproject.toml-based projects

[notice] A new release of pip is available: 24.0 -> 24.2
[notice] To update, run: pip install --upgrade pip

Versions

Operating System Information Darwin MadMax.local 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct 9 21:33:00 PDT 2023; root:xnu-10002.41.9~7/RELEASE_ARM64_T6031 arm64

Python Version Python 3.11.8

PIP Version pip 24.0 from /Users/jessewhite/Documents/source/torchchat/.venv/lib/python3.11/site-packages/pip (python 3.11)

Installed Packages absl-py==2.1.0 accelerate==0.34.0 aiohappyeyeballs==2.4.0 aiohttp==3.10.5 aiosignal==1.3.1 altair==5.4.1 annotated-types==0.7.0 anyio==4.4.0 attrs==24.2.0 blinker==1.8.2 blobfile==3.0.0 cachetools==5.5.0 certifi==2024.8.30 chardet==5.2.0 charset-normalizer==3.3.2 click==8.1.7 cmake==3.30.2 colorama==0.4.6 DataProperty==1.0.1 datasets==2.21.0 dill==0.3.8 distro==1.9.0 evaluate==0.4.2 filelock==3.15.4 Flask==3.0.3 frozenlist==1.4.1 fsspec==2024.6.1 gguf==0.10.0 gitdb==4.0.11 GitPython==3.1.43 h11==0.14.0 httpcore==1.0.5 httpx==0.27.2 huggingface-hub==0.24.6 idna==3.8 itsdangerous==2.2.0 Jinja2==3.1.4 jiter==0.5.0 joblib==1.4.2 jsonlines==4.0.0 jsonschema==4.23.0 jsonschema-specifications==2023.12.1 lm_eval==0.4.2 lxml==5.3.0 markdown-it-py==3.0.0 MarkupSafe==2.1.5 mbstrdecoder==1.1.3 mdurl==0.1.2 more-itertools==10.4.0 mpmath==1.3.0 multidict==6.0.5 multiprocess==0.70.16 narwhals==1.6.0 networkx==3.3 ninja==1.11.1.1 nltk==3.9.1 numexpr==2.10.1 numpy==1.26.4 openai==1.43.0 packaging==24.1 pandas==2.2.2 pathvalidate==3.2.1 peft==0.12.0 pillow==10.4.0 portalocker==2.10.1 protobuf==5.28.0 psutil==6.0.0 pyarrow==17.0.0 pybind11==2.13.5 pycryptodomex==3.20.0 pydantic==2.8.2 pydantic_core==2.20.1 pydeck==0.9.1 Pygments==2.18.0 pytablewriter==1.2.0 python-dateutil==2.9.0.post0 pytz==2024.1 PyYAML==6.0.2 referencing==0.35.1 regex==2024.7.24 requests==2.32.3 rich==13.8.0 rouge_score==0.1.2 rpds-py==0.20.0 sacrebleu==2.4.3 safetensors==0.4.4 scikit-learn==1.5.1 scipy==1.14.1 sentencepiece==0.2.0 six==1.16.0 smmap==5.0.1 snakeviz==2.2.0 sniffio==1.3.1 sqlitedict==2.1.0 streamlit==1.38.0 sympy==1.13.1 tabledata==1.3.3 tabulate==0.9.0 tcolorpy==0.1.6 tenacity==8.5.0 threadpoolctl==3.5.0 tiktoken==0.7.0 timm==1.0.7 tokenizers==0.19.1 toml==0.10.2 tomli==2.0.1 torch==2.5.0.dev20240716 torchao @ git+https://github.com/pytorch/ao.git@e11201a62669f582d81cdb33e031a07fb8dfc4f3 torchaudio==2.4.0.dev20240716 torchsr==1.0.4 torchvision==0.20.0.dev20240716 tornado==6.4.1 tqdm==4.66.5 tqdm-multiprocess==0.0.11 transformers==4.42.4 typepy==1.3.2 typing_extensions==4.12.2 tzdata==2024.1 urllib3==2.2.2 Werkzeug==3.0.4 word2number==1.1 xxhash==3.5.0 yarl==1.9.7 zstandard==0.23.0 zstd==1.5.5.1

PyTorch Version 2.5.0.dev20240716

dbort commented 1 week ago

Does the /Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/buck-out/v2 directory exist?

Does it help if you run

/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/buck2-bin/buck2-99773fe6f7963a72ae5f7b737c02836e kill

?

dbort commented 1 week ago

@byjlw said that the directory is present, and that the buck2 kill fixed it.

Thing is, we already have this line that's supposed to kill it when starting cmake: https://github.com/pytorch/executorch/blob/97396091080d152a09c20d4fa5a7ef3981b2ed48/build/Utils.cmake#L267

And the logs show that it's running: " -- Killing buck2 daemon". But obviously that didn't help here.

byjlw commented 1 week ago

Running this command fixed the problem.

/Users/jessewhite/Documents/source/torchchat/et-build/src/executorch/pip-out/temp.macosx-10.9-universal2-cpython-311/cmake-out/buck2-bin/buck2-99773fe6f7963a72ae5f7b737c02836e kill