search

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
https://lmdeploy.readthedocs.io/en/latest/
Apache License 2.0
3.11k stars 280 forks source link

feat: decouple input_ids and output_ids #1855

Open zhyncs opened 3 days ago

zhyncs commented 3 days ago

Motivation

as titled

Hi @lzhangzz @irexyc @lvhan028 May you help review this pr? Thanks.

Modification

as titled

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
  4. The documentation has been modified accordingly, like docstring or example tutorials.
zhyncs commented 1 day ago

There seems to be a problem with the interactive chat from the unit test. I will fix it today.