microsoft / teams-ai

SDK focused on building AI based applications and extensions for Microsoft Teams and other Bot Framework channels
MIT License
415 stars 182 forks source link

[Dev support]: Is it possible to generate an Adaptive Card response using prompt only #2041

Closed asith-w closed 3 weeks ago

asith-w commented 3 weeks ago

Question

I am using the .NET library and would like to know if I can generate a small summary of a user-requested topic directly from a prompt and include it in an Adaptive Card, as shown in the attached image.

Is this feasible ? I tried it but didn't work as expected.

image

Here is my prompt

You are an AI assistant for Microsoft Teams Bot

When a user asks a question about a specific topic, the You should generate a brief summary and include it in the below adaptive card before sending it

here is the adaptive card response for the teams

{ "type": "AdaptiveCard", "speak": "Apricot-chile glazed chicken recipe", "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "version": "1.5", "body": [ { "type": "Image", "altText": "Image of Apricot-Chile Glazed Chicken", "size": "Stretch", "url": "https://onedrive.live.com/embed?resid=EBE108865B49234E%21142072&authkey=%21AIH0yXnxgH5RoCY&width=840&height=360" }, { "type": "TextBlock", "text": "Title for user input", "wrap": true, "size": "Large", "weight": "Bolder", "color": "Default" }, { "type": "TextBlock", "id": "truncatedText", "text": "AI generated summary for input. AI generated summary for input.AI generated summary for input.AI generated summary for inputAI generated summary for input.AI generated summary for input", "wrap": true, "maxLines": 3 }, { "type": "RichTextBlock", "id": "showMore", "targetWidth": "atLeast:narrow", "inlines": [ { "type": "TextRun", "text": "Show more", "selectAction": { "type": "Action.ToggleVisibility", "targetElements": [ "truncatedText", "fullText1", "fullText2", "showMore", "showLess" ] } } ] }, { "type": "ActionSet", "targetWidth": "atLeast:narrow", "actions": [ { "type": "Action.OpenUrl", "title": "Save Summary", "url": "https://www.tasteofhome.com/recipes/spicy-apricot-glazed-chicken/" }, { "type": "Action.Submit", "title": "Bact to Cart", "iconUrl": "https://raw.githubusercontent.com/OfficeDev/Microsoft-Teams-Card-Samples/main/samples/recipe/assets/addToCart_icon.png" } ], "spacing": "Medium" } ] }

SubbaReddi commented 3 weeks ago

@asith-w : It is possible but sending card must be handled in code. Please check following sample app for reference: https://github.com/microsoft/teams-ai/tree/main/dotnet/samples/04.ai.f.vision.cardMaster

asith-w commented 3 weeks ago

Hi @SubbaReddi,

Thank you for your response. I tried the sample and created my own action similar to [Action("SendCard")]. However, I noticed that this action is being called multiple times for a single user input. Is there a way to prevent this from happening?

video of the issue https://github.com/user-attachments/assets/1584b726-3213-4cff-971a-558fbe19e1b1

SubbaReddi commented 3 weeks ago

@asith-w : Help me with repro steps to validate the issue. because i see issue is not reproducible with SendCard.

asith-w commented 3 weeks ago

The only change I made to the sample was adding a new action method called ShowBriefSummary ShowBriefSummary is hitting many times per command.

 [Action("ShowBriefSummary")]
 public async Task<string> ShowSummary([ActionTurnContext] ITurnContext turnContext, [ActionTurnState] AppState turnState, [ActionParameters] Dictionary<string, object> args)
 {
     if (args.TryGetValue("briefSummary", out object? cardObject) && cardObject is string summary)
     {
         await turnContext.SendActivityAsync($"<pre>{summary}</pre>");
         return "action is done";
     }

     return "failed to parsed card from action arguments";
 }

actions.json skprompt.txt

asith-w commented 3 weeks ago

It seems like the issue was related to the history feature. After I changed includeHistory(actions.json) to false, it worked without any problems