search

astral-sh / rye

a Hassle-Free Python Experience
https://rye.astral.sh
MIT License
13.6k stars 466 forks source link

Cannot install with build args while UV can #1271

Open this-josh opened 2 months ago

this-josh commented 2 months ago

I originally had this as a discussion but as UV works it seems like a valid issue.

I need llama-cpp-python with cuda, according the installation docs I need to run CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python

I've tried CMAKE_ARGS="-DGGML_CUDA=on" rye add llama-cpp-python and this doesn't work, I've also tried setting export CMAKE_ARGS=-DGGML_CUDA=on and including this in a .env file and then running rye add llama-cpp-python. All of these approaches give me the CPU only version

However, this works

rye add pip
source .venv/bin/activate
CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python

Also this works

rye add uv
CMAKE_ARGS="-DGGML_CUDA=on" uv pip install llama-cpp-python

Originally posted by @this-josh in https://github.com/astral-sh/rye/discussions/1270

this-josh commented 2 months ago

A similar question was asked by @Fissium in https://github.com/astral-sh/rye/discussions/469

mikefaille commented 3 weeks ago 
export FORCE_CMAKE="1" CMAKE_ARGS="-DGGML_CUDA=ON -DCMAKE_CXX_COMPILER=/usr/bin/g++-12"
uv clean llama-cpp-python
rye add llama-cpp-python

Explanation:

FORCE_CMAKE="1" : Ensures that CMake is re-run even if existing build files are present, which is necessary when changing compiler settings. CMAKE_ARGS="-DGGML_CUDA=ON -DCMAKE_CXX_COMPILER=/usr/bin/g++-12" : -DGGML_CUDA=ON : Enables CUDA support in the build process. -DCMAKE_CXX_COMPILER=/usr/bin/g++-12 : Explicitly sets the C++ compiler to g++-12. This is required because your version of nvcc doesn't fully support GCC 13 yet, and you need to use the older g++-12 for compatibility. Note: The NVCC_PREPEND_FLAGS approach is not the most reliable way to set the host compiler in newer CUDA versions. Directly setting CMAKE_CXX_COMPILER is the preferred method.

➜ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Aug_14_10:10:22_PDT_2024
Cuda compilation tools, release 12.6, V12.6.68
Build cuda_12.6.r12.6/compiler.34714021_0

gcc version : 

➜ gcc --version
gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

cmake version : 

➜ cmake --version
cmake version 3.28.3