[Dev support]: Token Limit Issue When Summarizing Large Planner Task JSON in Teams AI .NET Bot

[asith-w]

Question

I'm developing an AI bot using the Teams AI .NET library to summarize Microsoft Planner task details.
The bot retrieves task data via the Microsoft Graph API and summarizes the response JSON. However, I'm hitting token limit issues for prompt due to the large size of the JSON payload. (I use action for api request)) I know this is relevant to Open AI . but ,are there any alternative approaches within the Teams AI library to handle this efficiently

Code snippets

 [Action("GetMyTasks")]
 public async Task<string> GetMyTasks([ActionTurnContext] ITurnContext turnContext, [ActionTurnState] AppState turnState, [ActionParameters] Dictionary<string, object> parameters)
 {
    //call graph api
    //--
    return response
 }  

[corinagum]

What are your token limits set to in your planner's config.json? Have you tried increasing that?

[asith-w]

{ "schema": 1.1, "description": "A bot that can manage tasks summarization", "type": "completion", "completion": { "model": "gpt-4o", "completion_type": "chat", "include_history": true, "include_input": true, "max_input_tokens": 4000, "max_tokens": 1000, "temperature": 0.2, "top_p": 0.0, "presence_penalty": 0.6, "frequency_penalty": 0.0, "stop_sequences": [] }, "augmentation": { "augmentation_type": "monologue" } }

[corinagum]

For a summarization app, 1000 max_tokens seems quite small. Please try increasing your max_tokens, but also note that increased token usage will likely result in a higher bill from OpenAI.