issues
search
ztxz16
/
fastllm
纯c++的全平台llm加速库,支持python调用,chatglm-6B级模型单卡可达10000+token / s,支持glm, llama, moss基座,手机端流畅运行
Apache License 2.0
3.28k
stars
332
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
ChatGLM3-6B, 导出flm模型后,报浮点数例外(核心已转储)
#385
xinaiwunai
closed
8 months ago
3
修复部分系统的挂起问题,支持在Windows下使用Python模块
#384
TylunasLi
closed
8 months ago
0
Update chatglm_export.py
#383
xinaiwunai
closed
8 months ago
0
用root用户cmake 和 make 之后,用应用用户"ai"进行python setup.py install 报权限不足
#382
xinaiwunai
closed
8 months ago
2
Fix 修复windows下python简易接口不能加载fastllm_tools.dll
#381
helloimcx
closed
8 months ago
2
tokenizer对不齐
#380
eigen2017
closed
4 months ago
2
支持国产GPU卡
#379
wangyumu
closed
8 months ago
1
PEFT LOAR Segmentation fault (core dumped)
#378
chestnut111
opened
9 months ago
3
qwen模型回复缺字
#377
hediyuan
opened
9 months ago
1
某次Merge更新后(约10月30号前后),所有flm模型的推理质量出现大幅度下降,怀疑有显存提前被释放了,这里以chatglm3为例
#376
ColorfulDick
opened
9 months ago
1
Floating point exception
#375
wenweichan
closed
9 months ago
1
chatglm3-6b-32k的位置编码扩展方式似乎与chatglm2-6b-32k不同
#374
TylunasLi
closed
7 months ago
4
道歉贴:我因为fork了此项目,被人一直辱骂和骚扰。致此向原作者道歉。
#373
ArtificialZeng
closed
6 months ago
2
chatglm2 跑完以后 释放内存报错: CUDA error when release memory
#372
xiaoduozhou
opened
9 months ago
2
ptuning 训练后的chatGLM2 可以用fastllm吗
#371
xiaoduozhou
opened
9 months ago
4
修复Pyfastllm的命令行示例并更新文档
#370
TylunasLi
closed
9 months ago
0
加速chatglm2感觉没效果,和pytho直接调用都差不多是 30ms/token
#369
17714196157
opened
9 months ago
1
ModuleNotFoundError: No module named 'pyfastllm'
#368
Cloopen-ReLiNK
opened
9 months ago
3
make -j编译出错
#367
DreamTeamWangbowen
opened
10 months ago
4
Request for Support for Ascend Series Graphics Cards
#366
junior-zsy
opened
10 months ago
1
两个与ToDevice有关的优化
#365
TylunasLi
closed
9 months ago
0
修复GLM模型内存溢出问题
#364
fluxlinkage
closed
10 months ago
0
Error: CUDA error when release memory! state 4.
#363
htSun1998
closed
10 months ago
2
Fixes #360 add stop_token_ids
#362
wangyumu
closed
10 months ago
0
增加一个stop_token_ids可选参数,解决推理停不下来问题
#361
wangyumu
closed
10 months ago
1
微调后推理停不下来的问题
#360
wangyumu
closed
10 months ago
1
add support macos
#359
purpleroc
closed
10 months ago
1
ChatGLM3开源了,请问作者有支持计划吗?
#358
micrazy
opened
10 months ago
16
如果用gpu编译这个项目后,用fastllm加载模型是只能用gpu推理嘛
#357
dingli06
opened
10 months ago
2
ChatGLM2-6B 加速后模型返回空
#356
kexin-L
opened
10 months ago
0
单卡多模型能实现推理并行吗
#355
780966854
opened
10 months ago
0
代码结构优化
#354
wildkid1024
closed
10 months ago
0
Win32Demo工程增加GLM模型
#353
TylunasLi
closed
10 months ago
0
pyfastllm 内存泄露
#352
eric642
opened
10 months ago
7
更新llama.cpp,应用一些chatglm.cpp中的优化
#351
siemonchan
closed
10 months ago
0
求助求助,llama模型生成第一个token时,有3个代码块耗时严重【已解决2个,仅剩1个🙏🙏🙏】
#350
AnShengqiang
opened
10 months ago
10
增加释放权重所占内存空间的接口
#349
siemonchan
closed
10 months ago
0
GPU资源调用异常
#348
LivinLuo1993
opened
10 months ago
0
千问7B转换运行报错cublas error.terminate called after throwing an instance of 'char const*'
#347
wujingbo-web
closed
10 months ago
1
修复wheel构建
#346
wildkid1024
closed
10 months ago
2
提升torch2flm转换时4bit模型的量化精度
#345
siemonchan
closed
11 months ago
0
修复 pybind11 submodule 被破坏导致pyfastllm构建失败
#344
TylunasLi
closed
11 months ago
0
增加baichuan2转换脚本
#343
siemonchan
closed
11 months ago
0
是否考虑支持baichuan2-7B的RoPE编码?
#342
91he
closed
11 months ago
3
修复transformers>=4.33.0时,activate_adapter作为方法出现在model中的问题
#341
siemonchan
closed
11 months ago
0
baichuan2flm.py ..python3.10/site-packages/accelerate/big_modeling.py", line 415, in wrapper raise RuntimeError("You can't move a model that has some modules offloaded to cpu or disk.") RuntimeError: You can't move a model that has some modules offloaded to cpu or disk.
#340
zxzxde
opened
11 months ago
2
/home/jwkj/miniconda3/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found
#339
zxzxde
opened
11 months ago
1
兼容ChatGLM-6B最初版本,并修正ChatGLM2-6B的prompt构造问题
#338
TylunasLi
closed
11 months ago
0
batch_response() 耗时和prompt list长度成线性关系
#337
Lzhang-hub
opened
11 months ago
9
chatglm2微调后的模型能加速 并且部署吗
#336
sssssshf
opened
11 months ago
2
Previous
Next