search

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
https://lmdeploy.readthedocs.io/en/latest/
Apache License 2.0
4.33k stars 390 forks source link

[Feature] support edge chips #1767

Closed PredyDaddy closed 3 months ago

PredyDaddy commented 3 months ago

Motivation

My project only allow me to used orin series. But seems lmdeploy does not support this. Can I know if this is on the way?

Related resources

No response

Additional context

No response

lvhan028 commented 3 months ago

Can this project do you the favor?

PredyDaddy commented 3 months ago

Thanks for reply. This is an amazing project! But I am looking for vlm deploy on Jeson actually. Do you have any recommend?

BestAnHongjun commented 3 months ago

We will make every effort to adapt LMDeploy v0.4.2 to the Jetson Orin series in the near future. Thank you for your interest in the LMDeploy-Jetson project!