To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
The default argument for the 'use_llmlingua2' parameter is False, but the docstring is incorrectly written as True. The DOCUMENT.md file is correctly written as False.
What does this PR do?
The default argument for the 'use_llmlingua2' parameter is
False
, but the docstring is incorrectly written asTrue
. TheDOCUMENT.md
file is correctly written asFalse
.https://github.com/microsoft/LLMLingua/blob/40ac969a82f162b3eb0b8e1f1416756d442e4eec/DOCUMENT.md?plain=1#L283
Before submitting
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.
@iofu728 @mydmdm