Closed Dakraid closed 7 months ago
Hi @Dakraid, I deeply appreciate your efforts and for pointing out the bug. This issue has now been fixed, and we will release a new pip package as soon as possible.
You can update to the latest version by using pip install git+https://github.com/microsoft/LLMLingua.git
.
Hello, I've recently created a small service that intercepts calls from my frontend to my LLM backend to compress the incoming prompts. To retain some crucial parts, I wanted to make use of the force_context_ids, but as it turns out, I'm getting an error I can't further investigate.
I use this method to search the list of strings for the context parameter for matches against the regex and add them to a list of integers that represent their position in that first list. The list of integers is then handed to the compress_prompt() call.
The full code can be found here: https://github.com/Dakraid/LLMLinguaMITM/blob/main/main.py
During debugging, the list of integers seemed fine and matched the expected positions in the list of strings for the context. Any further help or insight would be appreciated.