issues
search
ztxz16
/
fastllm
纯c++的全平台llm加速库,支持python调用,chatglm-6B级模型单卡可达10000+token / s,支持glm, llama, moss基座,手机端流畅运行
Apache License 2.0
3.23k
stars
325
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
fix docker build error, update cmake version and base image
#430
peter4431
closed
4 months ago
0
ResponseBatch 返回结果不正确
#429
Liufeiran123
opened
4 months ago
5
MiniCPM模型Win32Demo工程编译、GPU执行问题修复
#428
TylunasLi
closed
4 months ago
0
batch padding mask 处理的相关代码
#427
Liufeiran123
closed
4 months ago
0
support export minicpm-2b-float16.flm
#426
hadoop2xu
closed
4 months ago
0
支持llama类模型的外推位置编码,增加Deepseek-Coder-Instruct和Qwen1.5-Chat模型支持
#425
TylunasLi
closed
4 months ago
0
fix minicpm
#424
hadoop2xu
closed
4 months ago
0
support OpenBMB/MiniCPM
#423
hadoop2xu
closed
4 months ago
0
如何贡献代码 ?
#422
hadoop2xu
closed
4 months ago
0
目前PEFT仅支持chatglm,什么时候可以支持其他模型,比如baichuan2呢?或者需要改哪些地方,很乐意contribute。
#421
jiahuanluo
opened
5 months ago
1
修复python脚本转换模型特殊token的错误
#420
TylunasLi
closed
5 months ago
0
后续能否支持ChatGLM3的多轮
#419
chenyangjun45
opened
5 months ago
2
qwen输出结果错误
#418
Liufeiran123
closed
4 months ago
1
完全对齐Tokenizer,支持InternLM-7B和XVERSE-7B
#417
TylunasLi
closed
5 months ago
0
请求支持Grouped Query Attention
#416
TylunasLi
closed
4 months ago
0
LLaMA类模型优化
#415
TylunasLi
closed
5 months ago
0
大佬 想问下 利用率只跑到60% 是什么情况?
#414
Chenhuaqi6
opened
5 months ago
2
转化模型格式(.bin->.flm)时
#413
ColorfulDick
opened
5 months ago
2
修复输出数据超长,且使用stream_response_raw的时候报错
#412
aofengdaxia
closed
5 months ago
0
修复输出数据超长,且使用stream_response_raw的时候报错。
#411
aofengdaxia
closed
5 months ago
0
修复输出数据超长,且使用stream_response_raw的时候报错。
#410
aofengdaxia
closed
5 months ago
1
当输出数据特别长的时候报错。
#409
aofengdaxia
closed
5 months ago
2
报告一个chatGLM3 function_call的bugs
#408
aofengdaxia
closed
6 months ago
1
使用fastllm推理得到的结果和transforers推理得到的结果不一样。
#407
aofengdaxia
closed
6 months ago
1
chatGLM6b保存CUDA error when release memory!
#406
aofengdaxia
closed
5 months ago
1
在macos intel平台上使用报错
#405
aofengdaxia
closed
6 months ago
3
请教下作者tokenzier encode和decode那部分有对应的python代码或者链接吗
#404
Zhiwei35
closed
4 months ago
1
fetch_response获取首个token耗时不稳定问题
#403
yiguanxian
opened
6 months ago
0
make_input和model.weight.tokenizer.encode会产生多余空格问题
#402
yiguanxian
opened
6 months ago
3
建议对python调用的model增加model.device 接口
#401
xinaiwunai
opened
6 months ago
1
兼容chatglm3-6b-32k,修复model.save()保存的模型tokenizer未对齐问题。
#400
TylunasLi
closed
6 months ago
0
chatglm3-6b-32k使用fastllm加速后无法推理
#399
JinXuan0604
opened
6 months ago
2
支持chatglm3-6b-32k,修复model.save()方式保存的模型没对齐的问题。
#398
TylunasLi
closed
6 months ago
0
flm的tokenizer和原始tokenizer分词结果不一致
#397
yiguanxian
opened
6 months ago
1
BAICHUAN2没有MakeInput的实现
#396
yiguanxian
closed
4 months ago
7
运行几个c++示例程序都直接报段错误
#395
1414945241
opened
6 months ago
1
Windows下Pyfastllm不能正常import
#394
helloimcx
closed
6 months ago
1
fastllm是否支持经过自己p-tuning v2后的chatGLM3-6b呢
#393
aofengdaxia
closed
6 months ago
1
修复非batch下CPU Attention算子取batch错误(#385)
#392
TylunasLi
closed
7 months ago
0
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
#391
weizhenhuan
opened
7 months ago
0
cmake 的时候提示找不到add_compile_definitions
#390
PeterXiaTian
opened
7 months ago
3
benchmark 测试的时候会卡住,如何解决呢?
#389
2213601279
opened
7 months ago
1
能详细介绍一下pyfastllm怎么用吗
#388
cstk2715
opened
7 months ago
0
pyfastllm有释放GPU显存的接口吗?
#387
hediyuan
opened
7 months ago
3
qwen 1.8b 输出结果错误
#386
suncheng-s
closed
4 months ago
0
ChatGLM3-6B, 导出flm模型后,报浮点数例外(核心已转储)
#385
xinaiwunai
closed
6 months ago
3
修复部分系统的挂起问题,支持在Windows下使用Python模块
#384
TylunasLi
closed
7 months ago
0
Update chatglm_export.py
#383
xinaiwunai
closed
7 months ago
0
用root用户cmake 和 make 之后,用应用用户"ai"进行python setup.py install 报权限不足
#382
xinaiwunai
closed
7 months ago
2
Fix 修复windows下python简易接口不能加载fastllm_tools.dll
#381
helloimcx
closed
7 months ago
2
Previous
Next