OpenPPL ppl.llm.serving issues

OpenPPL / ppl.llm.serving

Apache License 2.0

123 stars 13 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

支持qwen1.5或者qwen2吗？

#64 Flynn-Zh opened 1 month ago
1
How to generate custom dataset?

#63 trebladev opened 2 months ago
0
update git deps

#62 syheliel opened 3 months ago
1
Compilation error

#61 Liu-xiandong closed 3 months ago
3
关于性能分析的一点疑惑

#60 Zhiy-Zhang opened 4 months ago
1
[misc] move grpc related to src/serving/grpc

#59 ouonline closed 4 months ago
0
[fix] fix client of batch response

#58 Vincent-syr closed 4 months ago
0
fix batch rsp

#57 Vincent-syr closed 4 months ago
0
compile error：ppl.llm.serving/tools/client_pressure.cc:339:105: error: no matching function for call to ‘std::unique_ptr<grpc::ClientReader<ppl::llm::proto::Response> >::unique_ptr(std::unique_ptr<grpc::ClientReader<ppl::llm::proto::BatchedResponse> >)’

#56 Zhiy-Zhang closed 4 months ago
4
[feature] sever support token in & out, and batch rsp

#55 Vincent-syr closed 5 months ago
0
[feature] add early_stopping option

#54 Vincent-syr closed 5 months ago
0
[feature] add latency distribution for benchmark

#53 Vincent-syr closed 5 months ago
0
[refactor][doc] add README to internlm and chatglm, refactor model fa…

#52 Alcanderian closed 7 months ago
0
[feature] support pmx model

#51 xupinjie closed 7 months ago
0
[fix] fix sampler

#50 Alcanderian closed 8 months ago
0
[feature] support more cache layout and quant bit=0, quant group=1 for mlu

#49 Vincent-syr closed 10 months ago
0
Update build.sh

#48 Alcanderian closed 10 months ago
0
Update build.sh

#47 Alcanderian closed 10 months ago
0
[fix] fix memory leak error

#46 Vincent-syr closed 11 months ago
0
[fix] fix compile error with -DPPLNN_CUDA_ENABLE_NCCL=OFF

#45 Vincent-syr closed 11 months ago
0
Example demo stuck (ppl_llm_server/client_sample)

#44 Kwinpeng closed 11 months ago
2
[feature] add internlm model support and model, tokenizer factory

#43 Vincent-syr closed 11 months ago
0
[fix] qps benchmark fix

#42 Vincent-syr closed 11 months ago
0
[fix][doc] fix worker wait bug, and add llama config explain docs

#41 Vincent-syr closed 11 months ago
0
[feature] add `quant_method` option

#40 Alcanderian closed 11 months ago
0
test

#39 Vincent-syr closed 11 months ago
0
test

#38 ouonline closed 11 months ago
0
test

#37 Vincent-syr closed 11 months ago
0
编译出错 [ 17%] Built target crypto as: symbol lookup error: as: undefined symbol: deflate

#36 af-74413592 opened 11 months ago
2
适配 ppl.common 最新的逻辑

#35 Hijdk closed 11 months ago
3
[fix] fix 13b dead lock error

#34 Vincent-syr closed 11 months ago
0
[opt] reduce signal count

#33 ouonline closed 11 months ago
0
[feature] add simple flag, add request rate for benchmark

#32 Vincent-syr closed 11 months ago
0
[fix] add cache layout mode selecting, sampler deconstruct and misc

#31 Vincent-syr closed 11 months ago
0
[misc] fix simple error

#30 Vincent-syr closed 11 months ago
0
serving 运行出错

#29 maiquanshen closed 10 months ago
2
Syr/dev

#28 Vincent-syr closed 11 months ago
0
error: 服务器不允许请求未公开的对象 60209eb1ccc34d5deefb002d1b7f37545204f7f2 获取了子模组路径 'third_party/bloaty'，但是它没有包含 60209eb1ccc34d5deefb002d1b7f37545204f7f2。直接获取该提交失败。

#27 williamZQ closed 10 months ago
1
[fix] disable ppl.nn.llm build tools, disable

#26 Alcanderian closed 12 months ago
0
[fix] server dead lock bug

#25 Vincent-syr closed 12 months ago
0
[doc] update doc

#24 Alcanderian closed 12 months ago
0
[refactor][fix] use barrier to fix decoder and work thread sync, refa…

#23 Vincent-syr closed 12 months ago
0
[feature][refactor] support more tokenizer, optimize stream chat proc…

#22 Vincent-syr closed 12 months ago
0
Error for llama-13B on V100

#21 yisongsong opened 12 months ago
3
How to control the request rate in this framework?

#20 LRY89757 closed 12 months ago
3
How to disable kv cache 8-bit quantization?

#19 sleepwalker2017 closed 10 months ago
1
CMakeLists.txt:24 (hpcc_populate_dep) Configuring incomplete, errors occurred!

#18 LMX-xin closed 10 months ago
2
nccl error

#17 sleepwalker2017 closed 10 months ago
2
support for huggingface llama?

#16 sleepwalker2017 closed 10 months ago
3
[misc] ResourceManager::Init() changes and remove unused funcs

#15 ouonline closed 12 months ago
0