The llama-cpp-python installed using the following method cannot find llama.dll, but the actual llama.dll exists.

[chengjia604]

installation method： https://github.com/lperezmo/clblast-llama-cpp-python Download llama_cpp_python： error: Traceback (most recent call last): File "d:\python\Lib\site-packages\llama_cpp\llama_cpp.py", line 72, in _load_shared_library return ctypes.CDLL(str(_lib_path), **cdll_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\python\Lib\ctypes__init.py", line 376, in init__ self._handle = _dlopen(self._name, mode) ^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: Could not find module 'd:\python\Lib\site-packages\llama_cpp\llama.dll' (or one of its dependencies). Try using the full path with constructor syntax.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "d:\python\Lib\site-packages\llama_cpp\llama_cpp.py", line 86, in _lib = _load_shared_library(_lib_base_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "d:\python\Lib\site-packages\llama_cpp\llama_cpp.py", line 75, in _load_shared_library raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}") RuntimeError: Failed to load shared library 'd:\python\Lib\site-packages\llama_cpp\llama.dll': Could not find module 'd:\python\Lib\site-packages\llama_cpp\llama.dll' (or one of its dependencies). Try using the full path with constructor syntax.

[jreves]

I have the same issue, after finally getting a clean install with HIPBLAS. My standalone install of llama_cpp works, and runs on the GPU. Trying to import the library in jupyter - using this kernel - produces the same error as above. I also have llama.dll and llama.lib in my site-packages\llama_cpp directory.

Just running python interactively in my virtual environment, I am able to import llama without error.

[jreves]

I discovered something that might be useful and relevant for this issue: This error suggests that the dll can't be found, but it's obviously there in the correct location. The error is also generated if a dependency can't be found - and that was the case for me. I'm not sure how you compiled, but in my case I referenced the HIP code to access my AMD GPU. Turns out - I had not updated my windows system PATH to include the path to the directory for the dependent BLAS libraries. Adding those directories to my system path resolved this issue for me. Worth checking!

[abetlen]

@jreves that's correct, it's likely a dependency. If you check out llama_cpp.py line 58 actually adds the CUDA_PATH as a dll search directory, not sure if AMD has a similar common path variable but would be happy to add it if it helps.

[jreves]

@abetlen , yes there's a similar path for HIP installs on native Windows. The HIP directory is available as %HIP_PATH%, and the libraries are in %HIP_PATH$lib. The utilities - including the clang and clang++ compilers are in %HIP_PATH%bin.

%HIP_PATH% references the specific version as a part of the directory structure, so on my system - for example - %HIP_PATH$ expands to C:\Program Files\AMD\ROCm\5.5. The hipblas.dll file is in the bin directory.

There's a useful utility in the bin directory, as well: hipinfo.exe. Here's an example of the output: c:\Program Files\AMD\ROCm\5.5\bin>hipinfo

---

device# 0 Name: AMD Radeon RX 6900 XT pciBusID: 47 pciDeviceID: 0 pciDomainID: 0 multiProcessorCount: 40 maxThreadsPerMultiProcessor: 2048 isMultiGpuBoard: 0 clockRate: 2105 Mhz memoryClockRate: 1000 Mhz memoryBusWidth: 0 totalGlobalMem: 15.98 GB totalConstMem: 2147483647 sharedMemPerBlock: 64.00 KB canMapHostMemory: 1 regsPerBlock: 0 warpSize: 32 l2CacheSize: 4194304 computeMode: 0 maxThreadsPerBlock: 1024 maxThreadsDim.x: 1024 maxThreadsDim.y: 1024 maxThreadsDim.z: 1024 maxGridSize.x: 2147483647 maxGridSize.y: 65536 maxGridSize.z: 65536 major: 10 minor: 3 concurrentKernels: 1 cooperativeLaunch: 0 cooperativeMultiDeviceLaunch: 0 isIntegrated: 0 maxTexture1D: 16384 maxTexture2D.width: 16384 maxTexture2D.height: 16384 maxTexture3D.width: 2048 maxTexture3D.height: 2048 maxTexture3D.depth: 2048 isLargeBar: 0 asicRevision: 0 maxSharedMemoryPerMultiProcessor: 64.00 KB clockInstructionRate: 1000.00 Mhz arch.hasGlobalInt32Atomics: 1 arch.hasGlobalFloatAtomicExch: 1 arch.hasSharedInt32Atomics: 1 arch.hasSharedFloatAtomicExch: 1 arch.hasFloatAtomicAdd: 1 arch.hasGlobalInt64Atomics: 1 arch.hasSharedInt64Atomics: 1 arch.hasDoubles: 1 arch.hasWarpVote: 1 arch.hasWarpBallot: 1 arch.hasWarpShuffle: 1 arch.hasFunnelShift: 0 arch.hasThreadFenceSystem: 1 arch.hasSyncThreadsExt: 0 arch.hasSurfaceFuncs: 0 arch.has3dGrid: 1 arch.hasDynamicParallelism: 0 gcnArchName: gfx1030 peers: non-peers: device#0

memInfo.total: 15.98 GB memInfo.free: 15.86 GB (99%)

[abetlen]

@jreves perfect, I've added HIP_PATH to the dll search directories for windows users if it's defined in their environment. Should be in the next release

[abetlen]

@chengjia604 does that resolve your issue as well?

[QQCif]

I tried to use the prebuilt binary provided by llama.cpp repo. After setting LLAMA_CPP_LIB to the path of llama.dll I am getting the same error. cublas64_12.dll, cublasLt64_12, cudart64_12.dll are all present in the same folder as llama.dll. However if I tried to add llama.cpp binary folder to PATH, then it works.

[brockhager]

I'm having this same issue.

Here is the error I get: 02:08:15:llama_chat_adapter.py: from .llama_cpp import * 02:08:15:llama_chat_adapter.py: File "C:\Program Files\CodeProject\AI\modules\LlamaChat\bin\windows\python311\venv\lib\site-packages\llama_cpp\llama_cpp.py", line 88, in 02:08:15:llama_chat_adapter.py: _lib = _load_shared_library(_lib_base_name) 02:08:15:llama_chat_adapter.py: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 02:08:15:llama_chat_adapter.py: File "C:\Program Files\CodeProject\AI\modules\LlamaChat\bin\windows\python311\venv\lib\site-packages\llama_cpp\llama_cpp.py", line 77, in _load_shared_library 02:08:15:llama_chat_adapter.py: raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}") 02:08:15:llama_chat_adapter.py: RuntimeError: Failed to load shared library 'C:\Program Files\CodeProject\AI\modules\LlamaChat\bin\windows\python311\venv\lib\site-packages\llama_cpp\lib\llama.dll': Could not find module 'C:\Program Files\CodeProject\AI\modules\LlamaChat\bin\windows\python311\venv\lib\site-packages\llama_cpp\lib\llama.dll' (or one of its dependencies). Try using the full path with constructor syntax. 02:08:15:Module LlamaChat has shutdown

Local install on Win10 OS

I was trying to add an Environment Variable because I see the folder where the DLL files are.

They are here: C:\Program Files\CodeProject\AI\modules\LlamaChat\bin\windows\python311\venv\Lib\site-packages\llama_cpp\lib

Its like the server/app can't find that folder and DLL files and it just shuts down.