abetlen llama-cpp-python issues

abetlen / llama-cpp-python

Python bindings for llama.cpp

https://llama-cpp-python.readthedocs.io

MIT License

8.16k stars 970 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Fix: CUDA workflow actions

#1838 pabl-o-ce opened 12 hours ago
0
Error when update to 0.3.2

#1837 paoloski97 closed 2 days ago
0
Release the GIL

#1836 simonw opened 3 days ago
2
updated from 0.2.90 to 0.3.2 and now my GPU won't load

#1835 rookiemann opened 4 days ago
1
use n_threads param to call _embed_image_bytes fun

#1834 KenForever1 opened 5 days ago
0
Update dev instrunction

#1833 Florents-Tselai opened 1 week ago
0
Mistral-instruct not using system prompt.

#1832 AkiraRy opened 1 week ago
0
"eval time" and "prompt eval time" is 0.00ms after Ver0.3.0

#1830 nai-kon opened 1 week ago
0
save logits section in eval() sets dtype to np32 apparently unconditionally?

#1829 robbiemu closed 1 week ago
1
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='nul' mode='w' encoding='cp932'>

#1828 AkiraRy opened 1 week ago
0
Update related llama.cpp to support Intel AMX instruction

#1827 nai-kon closed 4 days ago
1
llama-server not using GPU

#1826 RakshitAralimatti opened 1 week ago
0
Update README.md

#1825 Nik-Kras opened 2 weeks ago
1
Request for prebuilt CUDA wheels for newer version

#1824 XJF2332 opened 2 weeks ago
7
Prebuilt CUDA wheels not working

#1822 mjwweb opened 2 weeks ago
2
chore(deps): bump conda-incubator/setup-miniconda from 3.0.4 to 3.1.0

#1821 dependabot[bot] opened 2 weeks ago
0
Feat: Support Ranking Method

#1820 yutyan0119 opened 2 weeks ago
0
docs: Remove ref to llama_eval in llama_cpp.py docs

#1819 richdougherty opened 2 weeks ago
0
AttributeError: function 'llama_sampler_init_tail_free' not found after compiling llama.pcc with hipBLAS

#1818 Micromanner opened 3 weeks ago
2
Support LoRA hotswapping and multiple LoRAs at a time

#1817 richdougherty opened 3 weeks ago
4
Specify GPU Selection (e.g., CUDA:0, CUDA:1)

#1816 RakshitAralimatti opened 3 weeks ago
4
Installed everything but speed is lower than on 3090 compared with industrial GPU. Seems like cuda is not working.

#1815 lukaLLM closed 1 week ago
3
[Feature request] High-level API support for DRY and XTC samplers

#1813 ddh0 opened 3 weeks ago
2
llama_get_logits_ith: invalid logits id -1, reason: no logits

#1812 ba0gu0 opened 3 weeks ago
1
Add support of Qwen2vl

#1811 PredyDaddy opened 3 weeks ago
2
Setting seed to -1 (random) or using default LLAMA_DEFAULT_SEED generates a deterministic reply chain

#1809 m-from-space opened 4 weeks ago
1
fix: make content not required in ChatCompletionRequestAssistantMessage

#1807 feloy opened 1 month ago
0
Assistant message with tool_calls and without content raises an error

#1805 feloy opened 1 month ago
1
Unable to pip install

#1804 chinthasaicharan closed 1 month ago
0
low level examples broken after [feat: Update sampling API for llama.cpp (#1742)]

#1803 mite51 opened 1 month ago
0
Fix type hint for `logit_bias` parameter

#1802 ddh0 opened 1 month ago
0
`Llama.from_pretrained` should work with `HF_HUB_OFFLINE=1`

#1801 davidgilbertson opened 1 month ago
0
fix: additional_files support glob patterns

#1800 xianml opened 1 month ago
0
fix: Avoid thread starvation on many concurrent requests by making use of asyncio to lock llama_proxy context

#1798 gjpower opened 1 month ago
0
top_p = 1 causes deterministic outputs

#1797 oobabooga opened 1 month ago
2
fix: added missing exit_stack.close() to /v1/chat/completions

#1796 Ian321 opened 1 month ago
1
Fix: add missing exit_stack.close() to end of /v1/completions endpoint

#1795 gjpower closed 1 month ago
1
Add reranking support

#1794 donguyen32 opened 1 month ago
4
Fix: Refactor Batching notebook to use new sampler chain API

#1793 lukestanley opened 1 month ago
1
Long Context Generation Crashes Google Colab Instance

#1792 kazunator opened 1 month ago
0
Fixed typo at line 340 of README.md

#1791 Victoran0 opened 1 month ago
4
chore(deps): bump pypa/cibuildwheel from 2.21.1 to 2.21.3

#1790 dependabot[bot] opened 1 month ago
0
Can't install with Vulkan support in Ubuntu 24.04

#1789 wannaphong closed 1 month ago
1
fix: chat API logprobs format

#1788 domdomegg opened 1 month ago
1
server: chat completions returns wrong logprobs model

#1787 domdomegg opened 1 month ago
1
server types: Move 'model' parameter to clarify it is used

#1786 domdomegg opened 1 month ago
0
llama-cpp-python 0.3.1 didn't use GPU(

#1785 artyomboyko closed 1 month ago
15
Tool parser cannot analysis tool calls string from qwen2.5.

#1784 hpx502766238 opened 1 month ago
0
chore(deps): bump pypa/cibuildwheel from 2.21.1 to 2.21.2

#1783 dependabot[bot] closed 1 month ago
1
Fix LLAVA_CPP_LIB creating empty path

#1782 navratil-matej opened 1 month ago
0