[Teams] Images are not accessible when are sent from MS Teams to the bot via message extensions

[mpelekh]

Images are not accessible when are sent from MS Teams to the bot via message extensions

Steps to reproduce: 1. Use the snipping tool anywhere on your screen. 2. Paste it onto a message in a channel.

3. Send. 4. Send this message to the bot via message extension.

Actual result: The attachments are empty. The bot can't handle the attachment since there is no metadata in the payload to the uploaded attachment. Expected result: The bot has access to the attachment in the payload and can handle it.

Notes:

• Message extension in the manifest file:

  "composeExtensions": [
    {
      "botId": "<MICROSOFT_BOT_ID>",
      "canUpdateConfiguration": false,
      "commands": [
        {
          "id": "add_message",
          "type": "action",
          "title": "Capture as highlight",
          "description": "Capture message highlight",
          "initialRun": false,
          "fetchTask": true,
          "context": [
            "message"
          ],
          "parameters": [
            {
              "name": "param",
              "title": "param",
              "description": ""
            }
          ]
        }
      ]
    }
  ]

• The payload sent by MS Teams. Pay attention that attachments are empty even that image was uploaded:

  {
     "value":{
        "commandId":"add_message",
        "commandContext":"message",
        "requestId":"d674153ceda543d58619030658f91108",
        "messagePayload":{
           "linkToMessage":"https://teams.microsoft.com/l/message/19:fe5615aacf9b...@thread.tacv2/1663935658113",
           "id":"1663935658113",
           "replyToId":null,
           "createdDateTime":"2022-09-23T12:20:58.113Z",
           "lastModifiedDateTime":null,
           "deleted":false,
           "subject":null,
           "summary":null,
           "importance":"normal",
           "locale":"en-us",
           "body":{
              "contentType":"html",
              "content":"<div><div class=\"copy-paste-block\" itemprop=\"copy-paste-block\"><img height=\"250\" src=\"https://us-api.asm.skype.com/v1/objects/0-eus-d18-d6c465628633680d11509cd821df636f/views/imgo\" style=\"vertical-align:bottom;width:250px;height:250px\" width=\"250.45537340619308\" id=\"0-eus-d18-d6c465628633680d11509cd821df636f\" itemscope=\"\" itemtype=\"http://schema.skype.com/AMSImage\"></div>\n</div>"
           },
           "from":{
              "device":null,
              "conversation":null,
              "user":{
                 "userIdentityType":"aadUser",
                 "id":"217f2395-3077-4ef3-ae65-…”,
                 "displayName":"Mykola Pelekh"
              },
              "application":null
           },
           "reactions":[
  
           ],
           "mentions":[
  
           ],
           "attachments":[
  
           ]
        },
        "context":{
           "theme":"default"
        }
     },
     "conversation":{
        "id":"19:fe5615aacf9b…@thread.tacv2;messageid=1663935658113"
     },
     "name":"composeExtension/fetchTask",
     "imdisplayname":"Mykola Pelekh"
  }

[anishprasad01]

Hi @mpelekh,

A few questions:

• What language are you using?
• What are the versions of the relevant packages in your project?
• How is this bot deployed?
• Does the behavior change depending on which Teams client or conversation you use?

[mpelekh]

Hi @anishprasad01 !

• What language are you using?

TypeScript.

• What are the versions of the relevant packages in your project?

botbuilder: 4.16.0 botframework-connector: 4.16.0 botframework-schema: 4.16.0

• How is this bot deployed?

The issue is reproducible for the bot application from AppSource as well as for the custom build application. The bot is deployed as a node.js web server built with restify (v. 8.6.1) framework.

• Does the behavior change depending on which Teams client or conversation you use?

No, it doesn't. The behavior is the same for both Web and for Desktop Teams clients and it doesn't depend on the conversation that is used.

Note: This issue is reproducible in any case when the message extension is used. You can use the "Steps to Reproduce" from the issue description and check the appropriate request in the network tab to see the payload on the message extension invoke.

[anishprasad01]

I was able to reproduce this behavior. The bot is creating the Hero Card by getting the image from messagePayload.body.content as sent by Teams, and the attachments are indeed empty. You cannot simply access the URL provided in body, for security.

Let me verify if this is expected behavior or not, as that will inform next steps.

[anishprasad01]

@mpelekh,

Based on what I have heard from Teams and this Teams documentation on messaging extensions, it appears that this functionality is expected behavior, and will not be changed. Further inquiries on this point should be directed to Teams, as we don't control this on the bot side.