langchain-ai / langchain

🦜🔗 Build context-aware reasoning applications
https://python.langchain.com
MIT License
95.35k stars 15.47k forks source link

community: add truncation params when an openai assistant's run is created #28158

Open LuisMSotamba opened 1 week ago

LuisMSotamba commented 1 week ago

Description: When an OpenAI assistant is invoked, it creates a run by default, allowing users to set only a few request fields. The truncation strategy is set to auto, which includes previous messages in the thread along with the current question until the context length is reached. This causes token usage to grow incrementally: consumed_tokens = previous_consumed_tokens + current_consumed_tokens.

This PR adds support for user-defined truncation strategies, giving better control over token consumption.

Issue: High token consumption.

vercel[bot] commented 1 week ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment | Name | Status | Preview | Comments | Updated (UTC) | | :--- | :----- | :------ | :------- | :------ | | **langchain** | ⬜️ Ignored ([Inspect](https://vercel.com/langchain/langchain/3DcYFUPHhSShDGy4HZTCCozqZvWE)) | [Visit Preview](https://langchain-git-fork-luismsotamba-community-assi-a6afe2-langchain.vercel.app) | | Nov 16, 2024 11:02pm |
LuisMSotamba commented 4 days ago

@ccurme I am mentioning you here since It was opened 5 days ago and it haven't been attended yet.