ztxz16 fastllm issues - Githubissues

ztxz16 / fastllm

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

Apache License 2.0

3.32k stars 340 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

C++支持直接读取Qwen2.5系列HF模型

#496 TylunasLi opened 1 day ago
0
modify readme

#495 forXuyx closed 2 weeks ago
0
如何配置参数使服务处理请求并发数最大化

#494 xiaoshizijiayou opened 1 month ago
0
error: no suitable user-defined conversion from "__half" to "__nv_bfloat16" exists

#493 xiaoshizijiayou opened 1 month ago
0
ModuleNotFoundError: No module named 'ftllm'

#492 mingyue0094 closed 1 month ago
4
如何通过参数方式直接加载adapter?

#491 xiaoshizijiayou opened 1 month ago
28
增加embed python api接口，bert模型增加normalize

#490 jiewlmrh closed 2 months ago
0
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xce in position 4411: invalid continuation byte

#489 huqiangDu opened 2 months ago
0
pytorch模型转flm模型Killedt

#488 scutzhe closed 2 months ago
1
加速llama3-sqlcoder-8b (Finetuned from model: [Meta-Llama-3-8B-Instruct])模型时，输出内容错误，全部是"!!!!!"

#487 Juvember opened 2 months ago
1
模型权重转化之后和原来的模型回答的内容不一致

#486 Whylickspittle opened 2 months ago
1
chatglm 失去 function calling 能力

#485 NingRiCheng opened 3 months ago
0
编译完之后运行模型时报错

#484 supercj92 closed 3 months ago
1
C++支持直接读取Deepseek Coder V1系列HF模型

#483 TylunasLi closed 3 months ago
0
请问一下国产显卡Ascend 910 and Hygon DCU如何安装fastllm？

#482 cgq0816 opened 3 months ago
1
GLM4-V-9B什么时候会出部署代码呢？

#481 GalSang17 opened 3 months ago
0
如何多卡部署

#480 longcheng183 opened 4 months ago
1
C++支持直接加载DeepSeek V2 Lite系列的HF模型

#479 TylunasLi closed 4 months ago
0
完善“支持的模型”文档

#478 TylunasLi closed 4 months ago
0
修改install.sh

#477 255-1 closed 4 months ago
0
修复Windows上MSVC无法成功编译的Bug。

#476 fluxlinkage closed 4 months ago
0
可以在CMake中主动指定CUDA计算能力

#475 fluxlinkage closed 4 months ago
2
支持将直接读取safetrensors得到的模型存为flm格式，并加载推理

#474 TylunasLi closed 4 months ago
1
对于int4g模型，增加对fp16输入的支持

#473 jiewlmrh closed 4 months ago
0
对于int8模型，增加对fp16输入的支持

#472 jiewlmrh closed 4 months ago
0
OSError: libcublas.so.ll: cannot open shared odject file: No such file or directory

#471 lichengyang666 opened 4 months ago
1
Meta-Llama-3-70B-Instruct

#470 longcheng183 opened 5 months ago
5
直接读取Llama3，Qwen2的HF模型，apiserver webui benchmark使用ChatTemplate

#469 TylunasLi closed 5 months ago
0
修复__restrict__导致的restrict is not allowed错误

#468 ColorfulDick closed 5 months ago
1
支持转换glm4-9b-chat模型

#467 TylunasLi closed 5 months ago
0
修复webui/apiserver的Windows编译，并支持直接读取HF模型

#466 TylunasLi closed 5 months ago
0
GLM-4-6B-Chat转换成flm格式后不能加载

#465 HofNature closed 5 months ago
5
修复Windows下的编译

#464 TylunasLi closed 5 months ago
0
H800 docker 编译, half类型转换编译报错

#463 ShadowTeamCN closed 5 months ago
1
请问什么时候支持GLM-4 ？

#462 Stupid-Ai closed 5 months ago
4
qwen1.5 int4模型回复出现解码问题：UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 72-73: invalid continuation byte

#461 zhang415 opened 5 months ago
0
Added English Translation of Readme

#460 Wheylop closed 5 months ago
0
make -j过程中报错

#459 AIlaowong opened 5 months ago
3
添加add_special_tokens选项，默认true，支持chatglm

#458 levinxo closed 6 months ago
1
请问现在支持deepseekv2量化吗

#457 fw2325 closed 5 months ago
1
Revert "添加add_special_tokens选项，默认true，支持chatglm模型"

#456 ztxz16 closed 6 months ago
0
解决，arm64 windows下编译报错

#455 dignfei closed 6 months ago
0
[CMakeFiles/Makefile2:100: CMakeFiles/pyfastllm.dir/all]

#454 ttaop opened 6 months ago
0
提供一个兼容OpenAI 接口的http server

#453 MistSun-Chen closed 6 months ago
1
结果返回一直是<unk>

#452 VincentLore opened 6 months ago
3
添加add_special_tokens选项，默认true，支持chatglm模型

#451 levinxo closed 6 months ago
1
chatglm3 相同提示词生成结果一致

#450 ttaop opened 6 months ago
0
采用向量化访存优化旧架构GPU性能

#449 TylunasLi closed 7 months ago
0
Do you have a plan to implement the CudaCatOp?

#448 dp-aixball opened 7 months ago
0
中文输入无法识别；webui打开的地址无法访问。

#447 Mihubaba closed 7 months ago
1