microsoft / botbuilder-dotnet

Welcome to the Bot Framework SDK for .NET repository, which is the home for the libraries and packages that enable developers to build sophisticated bot applications using .NET.
https://github.com/Microsoft/botframework
MIT License
872 stars 480 forks source link

Images in Microsoft Teams bot messages/Adaptive Cards not rendering #3261

Closed stephanbisser closed 4 years ago

stephanbisser commented 4 years ago

Version

4.7

Describe the bug

When the bot sends an Adaptive Card which includes an image, the image is not rendered in Microsoft Teams, but it is in the emulator. The same is true if the image is coming from a QnA Maker response. This is a huge problem, if cards are not rendering accordingly...

Expected behavior

Images in Teams (whether within Cards or just in a message) should render correctly

Screenshots

Adaptive Card Teams: image

Adaptive Card Emulator: image

QnA Maker response Teams: image

Adaptive Card: welcomeCard.zip

clearab commented 4 years ago

Hi - Can you please share the JSON of the card? You can also try using the Card editor in App Studio in Teams to get a more accurate idea of how a card will be rendered for that particular channel. You can also take a look at the reference for Adaptive Card formatting in Teams, available here: https://docs.microsoft.com/en-us/microsoftteams/platform/task-modules-and-cards/cards/cards-format

clearab commented 4 years ago

Here's an example of an Adaptive Card with an image that should render correctly in Teams: https://docs.microsoft.com/en-us/microsoftteams/platform/task-modules-and-cards/cards/cards-reference#adaptive-card

xieofxie commented 4 years ago

@clearab Thanks for the document! It says Images must be at most 1024×1024 and 1 MB and this solves a long ago problem for myself.

stephanbisser commented 4 years ago

thanks @xieofxie that was the problem with the image size. That solves the problem in Adaptive Cards. But the issue with "normal" bot messages where pictures are embedded via markdown isn't solved:

I have a QnA Maker response which looks like this:

Image

If I test that in the QnA Maker test pane I get this result which is fine:

image

If I test the bot in the webchat I get this result which is fine:

image

If I test the bot in Teams I get this which is NOT correct:

image

How to solve that?

xieofxie commented 4 years ago

@stephanbisser I just know this feature.. Thanks any way. I have tried and got the same result. Maybe @clearab could help us..

clearab commented 4 years ago

Hi - I'd be happy to assist, but I'll need to see the JSON of the card in order to determine what the issue is.

Please keep in mind that each client is responsible for rendering their card. Because of this there is no guarantee that cards will render identically across channels. The rendering of the card occurs in the client application - the bot service / bot framework doesn't have any direct control at that point.

EricDahlvang commented 4 years ago

The original .json for the card is in the file shared: welcomeCard.zip

This contains the following url: https://solvionaterdwinstorage.blob.core.windows.net/erdwinblobstore/erdwin_transparent.png which is a 4000x4000 image.

{
  "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
  "type": "AdaptiveCard",
  "version": "1.0",
  "body": [
    {
      "type": "TextBlock",
      "text": "Hi, ich bin Erdwin, dein intelligenter Assistent!",
      "weight": "Bolder",
      "size": "Medium",
      "wrap": true
    },
    {
      "type": "ColumnSet",
      "columns": [
        {
          "type": "Column",
          "width": "auto",
          "items": [
            {
              "type": "Image",
              "url": "https://solvionaterdwinstorage.blob.core.windows.net/erdwinblobstore/erdwin_transparent.png",
              "size": "Large",
              "style": "Person"
            }
          ]
        },
        {
          "type": "Column",
          "width": "stretch",
          "items": [
            {
              "type": "TextBlock",
              "text": "Erdwin",
              "weight": "Bolder",
              "wrap": true
            },
            {
              "type": "TextBlock",
              "spacing": "None",
              "text": "Dein intelligenter Assistent",
              "isSubtle": true,
              "wrap": true
            },
            {
              "type": "TextBlock",
              "text": "Ich bin **Erdwin**, dein *intelligenter Assistent*. Ich bin hier um dich in allen Belangen deines täglichen Arbeitens zu unterstützen. Hier sind meine Hauptkompetenzen aufgelistet:",
              "wrap": true
            }
          ]
        }
      ]
    },
    {
      "type": "FactSet",
      "facts": [
        {
          "title": "FAQ:",
          "value": "Ich kann dir viele häufig gestellte Fragen beantworten. Frag einfach los..."
        },
        {
          "title": "News:",
          "value": "Wenn du möchtest kann ich dir gerne die neuesten Intranet News anzeigen. *Wenn du dies tun möchtest schreib mir* **Zeige mir die News** *oder klicke unten auf den Button*"
        }
      ],
      "spacing": "Small"
    },
    {
      "type": "TextBlock",
      "text": "Solltest du Hilfe benötigen, schreib mir einfach **Hilfe** und ich zeige dir was ich alles tun kann.",
      "wrap": true
    }
  ],
  "actions": [
    {
      "type": "Action.Submit",
      "title": "Hilfe",
      "data": "Hilfe"
    }
  ]
}
stephanbisser commented 4 years ago

yes this is the correct json! is the image size too big?

clearab commented 4 years ago

I believe the issue is the image is too large. Images sent to/from a bot must be at most 1024×1024 and 1 MB.

stephanbisser commented 4 years ago

ok thanks for the info as I didn't find any restrictions on the docs

xieofxie commented 4 years ago

Hi @clearab I believe we forget the markdown part.. Image embedded via markdown (like ![Duck](https://aka.ms/Fo983c)) won't work in Teams. It only displays a broken image as show above.

I saw you are the PM of Teams so maybe you could contact Teams to test and verify this. Thanks!

clearab commented 4 years ago

@xieofxie No, that will not work. You need to use something like:

{ "type": "Image", "url": "https://solvionaterdwinstorage.blob.core.windows.net/erdwinblobstore/erdwin_transparent.png", "size": "Large", "style": "Person" }

stephanbisser commented 4 years ago

@clearab the string you mentioned here doesn't work:

I have this in my QnA Maker KB as an answer (like you mentioned)

image

And I get exactly this back:

image

So how can we use pictures in markdown language inside a Teams bot (as other channels support that)?

stephanbisser commented 4 years ago

and please do not close the issue if it's not resolved, thank you

huaxing-yuan commented 4 years ago

Hi @stephanbisser, what environment are you using ? did you restricted azure blob access only to your entreprise ? I have a similar issue with image in Teams (while image loaded correctly in Web).

After my investigation, when using Teams channel, image urls are replaced to: https://urlp.asm.skype.com/v1/url/content?url=[original_image_url]

images are not loaded because my Azure Storage blob allows only internal access. I have to make azure storage account publicly visible. now images are rendered, but our images are also exposed to internet.

clearab commented 3 years ago

It looks like we're running into a few things:

  1. There is a size limit on images. They must be < 1024 x 1024 and 1 MB
  2. Inside a card, you need to use the image format for that type of card (and not embed the image in text)
  3. In a simple text message you can either
    1. Add the image as an attachment
    2. Change the message format to either xml or markdown, and use that syntax for your message, including embedding the image.

@Wajeed-msft can you take a look at the above and make sure my summary is correct?

parasdesaicims commented 3 years ago

@clearab I am having a similar issue. The above-given solution doesn't seem to work.

damudao commented 3 years ago

@clearab

I have just flagged and detailed a similar problem here: https://github.com/MicrosoftDocs/msteams-docs/issues/3124

The image in the adaptivecard is hosted in a public blob-only access level Azure blob storage account. The image renders without issue in Teams Desktop and Teams WebBrowser, but no longer in Teams iOS. This is since April 12th. It had otherwise been working without issue for over three years.

If we switch the image location to a non-Microsoft url, eg https://via.placeholder.com/1000 then it renders fine in Teams iOS.

It's not an image size problem. The image dimensions and filesize are well, well within the limits. It's not a json payload problem either as the payload json hasn't been changed in years - and as mentioned works fine on Desktop/web, or if the image url changed to non-Microsoft location. It's not a Blob storage account configuration problem either as we have not actively changed any settings in about a year.

clearab commented 3 years ago

Thanks for letting us know - that definitely sounds like a bug in the Teams iOS client, so we'll let them work it from the Issue you opened against their repo.

clearab commented 3 years ago

Love the setup of the LogicApp + Webhook + Adaptive Card. Such a clean, easy way to send notifications to Teams.

GerritVisserFDMG commented 2 years ago

Experiencing almost the same problem here on OSX but also on Android. Difference is that it also works every now and then. Image is 256x256 and about 16kb. Despite the small size however, downloading the image can sometimes take a bit. This has to do with the fact that the small image offered to teams is derived from a bigger image under the hood. The processing time needed for this can make it slow.

And i'm thinking maybe too slow at times. What i see looks like something in teams giving up after a timeout.

GerritVisserFDMG commented 2 years ago

Realized too late that is issue is closed but i still would like to share what i've found in case anyone runs into the same problem. It seems that Teams ignores the arguments that are supplied in the url. So the ?size=Small that i was using was left out and so the image supplied was considered too big by Teams. The images that were shown simply had an original size that was within the limitations.

jeffpatton1971 commented 10 months ago

@clearab, I understand that this thread is closed, but I'm not sure if this is the right place to report an issue I'm facing with the desktop Teams app. The app doesn't seem to display images correctly, and I'm not sure if it's an issue with adaptive cards or with Teams itself. I use ADO pipelines to query GitHub repositories for updated Terraform modules, and I announce the diffs in a channel. However, on the desktop app, the logo (a 15k PNG) won't render, while on the mobile and web app, it displays correctly. Moreover, it's difficult to determine what the issue is, as some images will render while others won't. For example, I found one that's hosted on Bing that works, but Getty Images, Imgur images, and the icon in my storage account that's publicly accessible do not work.