microsoft / botframework-sdk

Bot Framework provides the most comprehensive experience for building conversation applications.
MIT License
7.5k stars 2.44k forks source link

Not able to get the attachment in the backend #6560

Closed rahulyadav97185966 closed 1 year ago

rahulyadav97185966 commented 1 year ago

Github issues should be used for bugs and feature requests. Use Stack Overflow for general "how-to" questions.

Version

V4

Describe the bug

Hi Team,

I developed a bot that is currently deployed in MS Teams, For one use case I need to attach an attachment in it, I am able to attach the document using the "attach icon", But when I copy-paste the document directly into a chat, then I am not able to do Since in the backend I am not able to get any response.

In the backend we are getting the response in stepContext.context.activity.attachments when I attach a document through the attachment icon, But when I Copy and Paste it into the chat stepContext.context.activity.attachments are getting as null.

Expected behavior

We should able to get the stepContext.context.activity.attachments in the backend.

dmvtech commented 1 year ago

I will look into this.

rahulyadav97185966 commented 1 year ago

okiee

dmvtech commented 1 year ago

I ran a quick test, and from Teams through ngrok, I have confirmed that they are differently-structrued in the attachment portion:

Using attach icon

{
  "attachments": [
    {
      "contentType": "application/vnd.microsoft.teams.file.download.info",
      "content": {
        "downloadUrl": "<DOWNLOADURL>",
        "uniqueId": "dc01d473-d1fa-43ae-9214-820b9aca2c58",
        "fileType": "jpg"
      },
      "contentUrl": "<CONTNENT URL>",
      "name": "<FILENAME>"
    },
    {
      "contentType": "text/html",
      "content": ""
    }
  ]
}

Using copy-paste

{
  "text": " \r\n\r\n ",
  "textFormat": "plain",
  "attachments": [
    {
      "contentType": "image/*",
      "contentUrl": "<CONTNENT URL>"
    },
    {
      "contentType": "text/html",
      "content": "<CONTENT HTML>"
    }
  ]
}

The clear difference being the contentType. The copy-paste route also has text and textFormat fields (as it's more than just file download info. It's a whole text/html message). Is your bot perchance handling it different because it's seeing it as a text message instead of an attachment received?

rahulyadav97185966 commented 1 year ago

Hi @dmvtech,

We are getting the response of the text or an attachment in the next step using stepContext.result

We are getting the attachments in stepContext.context.activity.attachments, this is an array that contains the content type, file Name, FileType, contentURL. And if the user added the text along with the attachment ( Using copy-paste), Then we are getting stepcontext.context.activity.attachments as null.

rahulyadav97185966 commented 1 year ago

@dmvtech Any Updates ?

dmvtech commented 1 year ago

Hi @rahulyadav97185966

Does the issue occur as well when running the following sample: https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-file-upload/csharp

Like if you break on this line, is attachments still null for the copy-paste scenario?: https://github.com/OfficeDev/Microsoft-Teams-Samples/blob/e3d01b6cfe77f026b788536fd98f0f2cc765cccd/samples/bot-file-upload/csharp/Bots/TeamsFileUploadBot.cs#L39

rahulyadav97185966 commented 1 year ago

Hi @dmvtech ,

image

We are getting this as null (HIghlighted in Yellow).

rahulyadav97185966 commented 1 year ago

@dmvtech Any update on the above ?

dmvtech commented 1 year ago

Hi @rahulyadav97185966

Apologies for the delay. Is turnContext.Activity.Attachments null at that point?

rahulyadav97185966 commented 1 year ago

@dmvtech Yes. turnContext.Activity.Attachments we received as null (when we copy paste screenshot).

dmvtech commented 1 year ago

@rahulyadav97185966

Does this exhibit regardless of what OS or version/edition of Teams you are running?

Can you please share the following information:

rahulyadav97185966 commented 1 year ago

@dmvtech

please find below details.

anthonyarmatasMS commented 1 year ago

Adding that this issue is happens when using the email channel as well. It is using Bot Framework Composer v2.1.2 and the Bot framework version is v4. something. I am not sure where to find that at this moment.

When sending an image from the clipboard with and without text nothing appears in attachments. image image image image

When sending an attachment image along side one copied from a clipboard, they both show up. image image

dmvtech commented 1 year ago

Hi @rahulyadav97185966 Sorry for the delay. I am checking a few more things and will get back to you shortly.

rahulyadav97185966 commented 1 year ago

@dmvtech Any updates ?

dmvtech commented 1 year ago

Hi @rahulyadav97185966

I am unable to reproduce this issue or find any potential cause. If you are using the same sample I mentioned, and using the normal input field, then any type of image attachment should come across as type message and should contain 1 or more attachments. I just ran new tests using; attach files, drag-and-drop and copy->paste. All 3 work, contain attachments, are of type message and therefore are available in OnMessageActivityAsync.

Can you try or test with version 4.18.1 or greater of the bot framework SDK?

dmvtech commented 1 year ago

@anthonyarmatasMS

I'm not sure about email, but I can check. Are you using a specific bot sample to reproduce this issue? You can check the bot version in the .csproj file. If running 4.10.x, please see if you can upgrade to 4.18.1 or greater.

dmvtech commented 1 year ago

@anthonyarmatasMS I can confirm that Activity.Attachments will be empty if the image is embedded in an email. It is sent in the body of the email, but not included in the activity. For email channel; you will only retrieve if it is an actual attachment and not embedded.