kvcache-ai ktransformers issues

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

Apache License 2.0

641 stars 31 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Installation requirements

#89 arthurv opened 1 day ago
1
[fix] Fix some gpu dequant function doesn't support multi gpu bug

#88 Azure-Tang opened 1 day ago
0
are marline and q4k totally equivalent?

#87 Eutenacity opened 1 day ago
1
typo fix: KMisrtal -> KMistral

#86 xhedit opened 2 days ago
0
Getting reasonable performance on dual RTX 3090 and 128gb

#85 myfrienderic opened 3 days ago
5
可以给出详细的硬件配置清单吗?

#84 qixing-ai opened 3 days ago
2
Use cond var to avoid busy loop

#83 sayap opened 3 days ago
1
Seg Fault on long replies

#82 matthusby closed 3 days ago
2
Fix backend

#81 chenht2022 closed 3 days ago
0
Busy loop in cpu_backend/task_queue.cpp keeps 1 thread at 100% CPU when queue is empty

#80 sayap opened 5 days ago
5
Is deepseek-ai/DeepSeek-V2.5 supported?

#79 AshD closed 4 days ago
9
Fix: Wrong type of token list returned by prefill_and_generate

#77 TKONIY opened 1 week ago
0
8-GPU configuration on L40 OOM

#76 fengyang95 closed 3 days ago
8
How can i run internlm2_5-7b-chat-1m in ktransformers?

#74 Ma1oneZhang closed 5 days ago
4
When the input token exceeds 4096, an error will occur.

#73 fengyang95 closed 1 week ago
4
Support IQ4_XS dequantize

#72 sayap closed 1 week ago
4
[fix] Fix qlen > chunk_size mask is none error

#71 Azure-Tang closed 1 week ago
0
UnboundLocalError: cannot access local variable 'chunck_mask' where it is not associated with a value

#70 fengyang95 closed 1 week ago
2
Missing pip packages flash_attn and wheel

#69 bitbottrap closed 1 week ago
2
What is the maximum input token size supported for DeepSeek V2?

#68 fengyang95 closed 3 days ago
1
[fix] fix bugs about Qwen2-57B, install requirement, DockerFile

#67 UnicornChan closed 2 weeks ago
0
docker container fails to start due to missing package 'uvicorn'

#66 sammcj closed 4 days ago
1
Would you support glm4-chat-1m

#65 choyakawa opened 2 weeks ago
1
docs: update long_context_introduction.md

#64 eltociear closed 2 weeks ago
0
[Fix] Fix problem that ktransformers cannot offload whole layer in cpu

#62 Azure-Tang closed 2 weeks ago
0
docker builds and pip install broken - No module named 'cpufeature'

#61 sammcj closed 2 weeks ago
5
fix(docs): fix broken link

#60 sammcj closed 2 weeks ago
0
[fix] Fix readme datas

#58 Azure-Tang closed 2 weeks ago
0
[feature] release 0.1.3

#57 UnicornChan closed 2 weeks ago
0
Update README.md

#56 hyx1999 closed 2 weeks ago
0
Add a instruction for configuring CUDA_HOME and CUDA_PATH to the install section of README.md.

#54 hyx1999 closed 2 weeks ago
2
Support for Mistral-Large-Instruct-2407-GGUF ？

#53 LIUKAI0815 closed 2 weeks ago
2
Fix: None for load config

#52 UnicornChan closed 3 weeks ago
0
[fix] f16 dequantize device ignored

#51 molamooo closed 3 weeks ago
0
How to properly disable offloading MoE layers to CPU?

#50 molamooo closed 2 weeks ago
5
More Efficient Layer Distribution for DeepSeek Coder v2 on Multiple GPUs and CPUs

#49 BGFGB opened 3 weeks ago
4
[fix] Fix bugs about static cache and server param;

#48 Azure-Tang closed 3 weeks ago
0
Can I run llama3.1 70b with rtx4090+64g ddr5 ram?

#47 codeMonkey-shin opened 3 weeks ago
1
[ENHANCEMENT] improve GPU utilization for multi-GPU

#46 ELigoP closed 3 weeks ago
1
Cannot run DeepSeek V2 Chat in server mode on 2 GPUs

#45 ELigoP closed 3 weeks ago
1
CUDA error: No kernel image is available for execution on the device

#44 Forsworns closed 3 weeks ago
2
Update install.sh

#43 RealLittleXian closed 3 weeks ago
4
Mixtral-8x7B-v0.1 GGUF file error

#42 RealLittleXian closed 3 weeks ago
1
[Update] Update README

#41 Azure-Tang closed 4 weeks ago
0
[fix] fix broken link

#40 Azure-Tang closed 4 weeks ago
0
[fix] fix broken link

#39 Azure-Tang closed 4 weeks ago
0
[update] README

#38 Azure-Tang closed 1 month ago
0
Ubuntu 24.04 GLIBCXX version fail

#37 ELigoP closed 3 weeks ago
3
Release v0.1.2

#36 UnicornChan closed 1 month ago
0
[update] Update readme; Add tutorial

#35 Azure-Tang closed 1 month ago
0