build failure on aarch64

[malv-c]

============================ CUDA_VERSION: 114

NVCC path: /usr/local/cuda-11.4/bin/nvcc GPP path: /usr/bin/g++ VERSION: g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 CUDA_HOME: /usr/local/cuda-11.4 CONDA_PREFIX: PATH: /tmp/exllama/exllama/bin:/home/polk/.pyenv/shims:/home/polk/.pyenv/bin:~/.local/bin:/M/bin:/bin:/snap/bin:/usr/games:/usr/local/games:/usr/local/cuda-11.4/bin:/usr/local/bin:/M/sbin:/sbin:/usr/local/bin:/usr/local/sbin LD_LIBRARY_PATH: /usr/local/cuda/targets/aarch64-linux/lib:/usr/local/cuda-11.4/targets/aarch64-linux/lib:/usr/local/cuda-11/targets/aarch64-linux/lib:/usr/local/lib/aarch64-linux-gnu:/lib/aarch64-linux-gnu:/usr/lib/aarch64-linux-gnu:/usr/lib/aarch64-linux-gnu/tegra-egl:/usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/fakechroot:/usr/lib/aarch64-linux-gnu/libfakeroot:/usr/local/lib:/usr/lib/aarch64-linux-gnu/tegra:/opt/nvidia/vpi2/lib64:

/usr/local/cuda-11.4/bin/nvcc -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -Xcompiler '-fPIC' --use_fast_math -Xptxas=-v -dc /tmp/bitsandbytes/csrc/ops.cu /tmp/bitsandbytes/csrc/kernels.cu -I /usr/local/cuda-11.4/include -I /tmp/bitsandbytes/csrc -I /include -I /tmp/bitsandbytes/include -L /usr/local/cuda-11.4/lib64 -lcudart -lcublas -lcublasLt -lcusparse -L /lib --output-directory /tmp/bitsandbytes/build In file included from /tmp/bitsandbytes/include/BinSearch.h:5, from /tmp/bitsandbytes/csrc/ops.cu:10: /tmp/bitsandbytes/include/SIMD.h:32:10: fatal error: emmintrin.h: Aucun fichier ou dossier de ce type 32 | #include | ^~~~~ compilation terminated. make: *** [Makefile:94 : cuda11x] Erreur 1

[blizzn]

solution? compilation for arch64 possible?

[malv-c]

hi compilation for arch64 possible? no : else i did not made an issue ;)

Le lun. 31 juil. 2023 à 19:40, blizzn @.***> a écrit :

solution? compilation for arch64 possible?

— Reply to this email directly, view it on GitHub https://github.com/TimDettmers/bitsandbytes/issues/548#issuecomment-1658861884, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESIHJORKXUOKDG5NXULM6TXS7UZVANCNFSM6AAAAAAZXHZ2AI . You are receiving this because you authored the thread.Message ID: @.***>

[blizzn]

i found a docker file for supposedly the nvidia container plus webui working. https://github.com/oobabooga/text-generation-webui/blob/main/docs/Docker.md here some troubleshooting https://github.com/TimDettmers/bitsandbytes/issues/221 by androiddrew

I was thinking that via docker it could work maybe

[malv-c]

thanks as my jetson orin dev kit died i will not test it ;)

Le mar. 1 août 2023 à 13:26, blizzn @.***> a écrit :

i found a docker file for supposedly the nvidia container plus webui working. https://github.com/oobabooga/text-generation-webui/blob/main/docs/Docker.md here some troubleshooting #221 https://github.com/TimDettmers/bitsandbytes/issues/221 by androiddrew

I was thinking that via docker it could work maybe

— Reply to this email directly, view it on GitHub https://github.com/TimDettmers/bitsandbytes/issues/548#issuecomment-1660121741, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESIHJJBNJO7UQYW4E256R3XTDRVBANCNFSM6AAAAAAZXHZ2AI . You are receiving this because you authored the thread.Message ID: @.***>

[swumagic]

OR you are Linux distribution (Ubuntu, MacOS, etc.)system ,AND CUDA Version: 11.X.

Bitsandbytes can support ubuntu.（yuhuang） 1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui 2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes

3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows

4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/TimDettmers/bitsandbytes/releases/download/0.41.0/bitsandbytes-0.41.0-py3-none-any.whl

Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）

[malv-c]

thanks but this jetson orin is now not even able to boot so the best thing in return to give is an advice DON'T BUY NVIDIA BEFORE 2028 Le dim. 12 nov. 2023 à 02:55, swumagic @.***> a écrit :

OR you are Linux distribution (Ubuntu, MacOS, etc.)system ,AND CUDA Version: 11.X.

Bitsandbytes can support ubuntu.（yuhuang） 1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui 2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes

3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows

4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/TimDettmers/bitsandbytes/releases/download/0.41.0/bitsandbytes-0.41.0-py3-none-any.whl

Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）

— Reply to this email directly, view it on GitHub https://github.com/TimDettmers/bitsandbytes/issues/548#issuecomment-1806973807, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESIHJM63G2FLOBZ5GG22B3YEAUATAVCNFSM6AAAAAAZXHZ2AKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBWHE3TGOBQG4 . You are receiving this because you authored the thread.Message ID: @.***>

[swumagic]

Bitsandbytes support windows.（yuhuang） 1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui 2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes

3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows

4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl

Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）

[swumagic]

thanks but this jetson orin is now not even able to boot so the best thing in return to give is an advice DON'T BUY NVIDIA BEFORE 2028 Le dim. 12 nov. 2023 à 02:55, swumagic @.> a écrit : 谢谢，但这个 Jetson Orin 现在甚至无法启动 因此，最好的回报就是建议 在 2028 年之前不要购买 NVIDIA 11月12日2023 à 02：55， swumagic @.> a écrit ： … ... OR you are Linux distribution (Ubuntu, MacOS, etc.)system ,AND CUDA Version: 11.X. Bitsandbytes can support ubuntu.（yuhuang） 1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui 2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes 3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows 4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/TimDettmers/bitsandbytes/releases/download/0.41.0/bitsandbytes-0.41.0-py3-none-any.whl Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310） — Reply to this email directly, view it on GitHub <#548 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESIHJM63G2FLOBZ5GG22B3YEAUATAVCNFSM6AAAAAAZXHZ2AKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBWHE3TGOBQG4 . You are receiving this because you authored the thread.Message ID: @.***>

your pc is Windows or liunx?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[FremyCompany]

FWIW, the reason this error message appears seems to be that bitsandbytes does not ship a release version for CUDA 12.4 on an ARM64 machine. The only way is to recompile bitsandbytes from source. If you are doing this in a Dockerfile, this can be done using the following commands:

RUN pip uninstall -y bitsandbytes
RUN export BNB_CUDA_VERSION=124
RUN cd /workspace && git clone https://github.com/TimDettmers/bitsandbytes.git && cd bitsandbytes/
RUN cd /workspace/bitsandbytes && pip install -r requirements-dev.txt
RUN cd /workspace/bitsandbytes && cmake -DCOMPUTE_BACKEND=cuda -S .
RUN cd /workspace/bitsandbytes && make
RUN cd /workspace/bitsandbytes && pip install -e .

If you run this on the device itself, I suppose it should be possible to use

pip uninstall -y bitsandbytes
export BNB_CUDA_VERSION=124
git clone https://github.com/TimDettmers/bitsandbytes.git && cd bitsandbytes/
pip install -r requirements-dev.txt
cmake -DCOMPUTE_BACKEND=cuda -S .
make
pip install -e .

I'm running IGX Orin OS v1.0 (GA) and this does not error on my end. Maybe this used to fail in release preview builds, but this seems to be fixed now.