search

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
https://lmdeploy.readthedocs.io/en/latest/
Apache License 2.0
4.17k stars 376 forks source link

Inquiry #2392

Open xiaoajie738 opened 2 weeks ago

xiaoajie738 commented 2 weeks ago

Can the models deployed by llmdeploy support gRPC access? Thank you for taking the time to answer despite your busy schedule.

lvhan028 commented 2 weeks ago

Our current plan doesn't include the support for gRPC interfaces. Would you mind sharing the motivation of using gPRC instead of RESTful API?