InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

https://lmdeploy.readthedocs.io/en/latest/

Apache License 2.0

4.33k stars 390 forks source link

Open CSEEduanyu opened 3 months ago

CSEEduanyu commented 3 months ago

https://github.com/InternLM/lmdeploy/blob/fbd294adaef24dd98aeaf99f9b2246167c723451/lmdeploy/vl/utils.py#L14 这样会有轻微diff 大多数都不影响第一个token 但logits还是有差异的最好还是改成image.save(buffered, format='PNG')@lvhan028

lvhan028 commented 3 months ago

1773 正在解决这个问题

CSEEduanyu commented 3 months ago

1773 正在解决这个问题

ok 大佬知乎私信你了方便交流下不实测了引擎的吞吐能力确实很强方便学习下调度和kernel这块的优化不

lvhan028 commented 3 months ago

我鲜少关注知乎私信，可以在discussion板块发起讨论，大家都可以参与。组内同事们更有经验