Outgoing webhook sends several requests in a row

viktor-shmigol commented 3 months ago

Type of issue

Feature doesn't work (bug)

Feedback

Hello,

We are encountering an issue with multiple POST requests being sent to our API.

We created an outgoing webhook in our team, and when we mention this webhook (e.g., @Teams Test 1 test message), it triggers 3 or more identical messages to our API.

Please see the example below:

We tested it using the Node.js example from this repository: Microsoft Teams Samples - Outgoing Webhook (Node.js). This example is suggested by the following article: Add an outgoing webhook.

Additionally, we tested it using a simple Ruby Grape API and encountered the same issue. The Teams outgoing webhook is being sent to our API multiple times.

class API::V1::IncomingWebhooks::Messages < Grape::API

  format :json

  helpers do

    def request_hmac_signature

      request.headers['authorization'].to_s.split[1]

    end

    def valid_hmac?

      serialized_payload_bytes = request.body.read.encode('utf-8')

      key_bytes = Base64.decode64(ENV['TEAMS_SECRET'])

      hmac_sha256 = OpenSSL::HMAC.digest('sha256', key_bytes, serialized_payload_bytes)

      calculated_hmac_value = Base64.encode64(hmac_sha256).strip

      request_hmac_signature == calculated_hmac_value

    end

  end

  namespace 'incoming_webhooks/:id' do

    resources :messages do

      desc 'Creates message'

      post do

        return unauthorized! unless valid_hmac?

        { type: 'message', text: 'Successfully received' }

      end

    end

  end

end

The result is the same

Is there any solution to fix it?

Thanks!

Page URL

https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-outgoing-webhook?tabs=urljsonpayload%2Cdotnet

Content source URL

https://github.com/MicrosoftDocs/msteams-docs/blob/main/msteams-platform/webhooks-and-connectors/how-to/add-outgoing-webhook.md

Author

@surbhigupta

Document Id

43e90995-bba4-86b6-1f1c-ab749e546a4b

sayali-MSFT commented 3 months ago

@viktor-shmigol - Thanks for reporting your issue. We will check this at our end and will get back to you.

sayali-MSFT commented 2 months ago

@viktor-shmigol -We have tried the above scenario on both web and desktop clients, and it is working fine on our end. Could you please check it with the Teams version listed below? Microsoft Teams version 24215.1000.3037.6292. The client version is 49/24061318413.

sayali-MSFT commented 2 months ago

@viktor-shmigol -We will test this issue on a Mac device and let you know the update. We have tested it on a Windows device, and it is working fine there.

viktor-shmigol commented 2 months ago

@sayali-MSFT I tested it in WEB/Desktop in Windows as well. I'm getting the same bug.

sayali-MSFT commented 2 months ago

@viktor-shmigol -Could you please share the client version?

Have you test this with the below version?

Microsoft Teams version 24215.1000.3037.6292. The client version is 49/24061318413.

viktor-shmigol commented 2 months ago

@sayali-MSFT The desktop version is 24193.1707.3028.4282 (24193.1707.3028.4282) The client version is 1415/24062724817.

I've just upgraded the WEB. The issue is still present.

sayali-MSFT commented 2 months ago

@viktor-shmigol - Glad to hear that your issue is resolved.

viktor-shmigol commented 2 months ago

@sayali-MSFT I'm sorry. it still sending double posts. There was some glitch. It's not resolved yet.

sayali-MSFT commented 2 months ago

Could you please share the logs so that we can investigate from our end?

viktor-shmigol commented 2 months ago

@sayali-MSFT Here is the logs from API. As you can see the API recieved two requests with interval in one second.

2024-08-08T13:44:17.163832645Z app[web.1]: 172.17.0.1 - - [08/Aug/2024:13:44:17 +0000] "POST /api/v2/incoming_webhooks/4fc47565-955f-4ad6-86b2-e942811b3db8/microsoft_teams/messages HTTP/1.1" 201 60 "https://teams.microsoft.com/" "-"
2024-08-08T13:44:18.377968879Z app[web.1]: 172.17.0.1 - - [08/Aug/2024:13:44:18 +0000] "POST /api/v2/incoming_webhooks/4fc47565-955f-4ad6-86b2-e942811b3db8/microsoft_teams/messages HTTP/1.1" 201 60 "https://teams.microsoft.com/" "-"

Also, I'm receiving these errors while receiving reply messages from Webhook in WEB app. As you can see I'm receiving two JS records as well.

v2/:1 Access to fetch at 'https://urlp.asm.skype.com/v1/url/content?url=https%3A%2F%2Fres.cdn.office.net%2Fteamsappdata%2Fevergreen-assets%2Fapps%2Foutcomingwebhook_largeimage.png&v=1' from origin 'https://teams.microsoft.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
async-entry-a009e2428bc2c2a4.js:43 

       GET https://urlp.asm.skype.com/v1/url/content?url=https%3A%2F%2Fres.cdn.office.net%2Fteamsappdata%2Fevergreen-assets%2Fapps%2Foutcomingwebhook_largeimage.png&v=1 net::ERR_FAILED
nR @ async-entry-a009e2428bc2c2a4.js:43
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
await in (anonymous)
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
await in (anonymous)
getImageAsDataUri @ services-toast-image-328dfca659f19d98.js:1
toastImageAsDataUri @ r_data-resolvers-toast-85f6fa3766e334cc.js:1
await in toastImageAsDataUri
Ie @ 552520-4ca55c4a2f58170e.js:1
he @ 552520-4ca55c4a2f58170e.js:1
pe @ 552520-4ca55c4a2f58170e.js:1
(anonymous) @ 552520-4ca55c4a2f58170e.js:1
fe @ 552520-4ca55c4a2f58170e.js:1
(anonymous) @ supermassive-incremental-link-198b87ba581865c6.js:1
Promise.then
(anonymous) @ supermassive-incremental-link-198b87ba581865c6.js:1
(anonymous) @ data-view-schema-d7889a531a8fa089.js:1
e @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
(anonymous) @ async-entry-a009e2428bc2c2a4.js:30
Promise.then
(anonymous) @ async-entry-a009e2428bc2c2a4.js:30
e @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
e @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
complete @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
t @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
_ @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
E @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
query @ async-entry-a009e2428bc2c2a4.js:30
nhe @ async-entry-a009e2428bc2c2a4.js:43
await in nhe
rhe @ async-entry-a009e2428bc2c2a4.js:43
(anonymous) @ async-entry-a009e2428bc2c2a4.js:43
b @ 548427-50836f59a12e2130.js:1
E @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
next @ 548427-50836f59a12e2130.js:1
b @ 548427-50836f59a12e2130.js:1
E @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
next @ 548427-50836f59a12e2130.js:1
b @ 548427-50836f59a12e2130.js:1
E @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
F @ 548427-50836f59a12e2130.js:1
next @ 548427-50836f59a12e2130.js:1
b @ 548427-50836f59a12e2130.js:1
E @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
next @ async-entry-a009e2428bc2c2a4.js:30
b @ 548427-50836f59a12e2130.js:1
E @ 548427-50836f59a12e2130.js:1
(anonymous) @ 548427-50836f59a12e2130.js:1
next @ 548427-50836f59a12e2130.js:1
_ @ 548427-50836f59a12e2130.js:1
m @ 548427-50836f59a12e2130.js:1
value @ 548427-50836f59a12e2130.js:1
(anonymous) @ async-entry-a009e2428bc2c2a4.js:30
onNext @ async-entry-a009e2428bc2c2a4.js:30
onSuccess @ async-entry-a009e2428bc2c2a4.js:30
onSuccess @ async-entry-a009e2428bc2c2a4.js:30
setResponseAndEndScenario @ async-entry-a009e2428bc2c2a4.js:30
reconcileResponse @ async-entry-a009e2428bc2c2a4.js:30
onMessageReceived @ async-entry-a009e2428bc2c2a4.js:30
Iv.onResponseReceived @ async-entry-a009e2428bc2c2a4.js:30
handleResponse @ async-entry-a009e2428bc2c2a4.js:30
handleResponse @ async-entry-a009e2428bc2c2a4.js:30
onMessageCallback @ async-entry-a009e2428bc2c2a4.js:30
(anonymous) @ async-entry-a009e2428bc2c2a4.js:30
d.onmessage @ async-entry-a009e2428bc2c2a4.js:30
v2/:6 Unhandled error/rejection {"isTrusted":true}

sayali-MSFT commented 2 months ago

@viktor-shmigol- These logs show that two POST requests to the same endpoint were made, and both were successful (HTTP status 201). If this behavior is unexpected, you might want to:

Check for Duplicate Requests: Ensure that the sender is not retrying or sending duplicate requests unintentionally.
Review Webhook Configuration: Verify that the webhook is correctly configured and that no duplicate webhooks are set up.
Inspect Request Payloads: Check if the payloads of the two requests are identical or if there is any difference.

Also try the below attached sample, just change the sharedSecret in app.js file. nodejs.zip

viktor-shmigol commented 2 months ago

@sayali-MSFT Hello, I'm sorry for the long response. I've installed and tested the nodejs app which you sent https://github.com/MicrosoftDocs/msteams-docs/issues/11424#issuecomment-2283743674 .

I'm still getting double requests to the API. See attached screenshots:

Here is the request payload:

We have only one outgoing webhook:

I'm testing it on WEB Version The client version is 1415/24071128825.

If you need to provide more details just let me know

sayali-MSFT commented 2 months ago

@viktor-shmigol -We have tested this scenario again, and it is working fine on both web and desktop. The client version is 1415/24081700411.

We are checking your issue with the engineering team and will let you know the update.

viktor-shmigol commented 1 month ago

@sayali-MSFT We are still having this issue. Were you able to test this issue with DEV team? Thanks!

sayali-MSFT commented 1 month ago

@viktor-shmigol -We have tested this issue on Web and Desktop client as well.

You have Microsoft Teams version 24231.512.3106.6573. The client version is 49/24081700413.

viktor-shmigol commented 1 month ago

@sayali-MSFT Did they debug backend part of their side using my log?(See screenhsot bellow). Should I provide something else? Maybe I need to return some specific json OR inject some header?

{ "type": "message", text: "Successfully received" }

I'm asking as we cannot resolve double post issue. We tested it in different ways with our API and demo nodejs API: https://github.com/MicrosoftDocs/msteams-docs/issues/11424#issuecomment-2283743674 . The teams API sends it twice or more times. We tried to remove channel, team and outgoing webhooks, nothing helps. We are using different platforms such as WEB, desctop, but we are having the same issue.

sayali-MSFT commented 1 month ago

@sayali-MSFT Did they debug backend part of their side using my log?(See screenhsot bellow). Should I provide something else? Maybe I need to return some specific json OR inject some header?
{ "type": "message", text: "Successfully received" }
I'm asking as we cannot resolve double post issue. We tested it in different ways with our API and demo nodejs API: #11424 (comment) . The teams API sends it twice or more times. We tried to remove channel, team and outgoing webhooks, nothing helps. We are using different platforms such as WEB, desctop, but we are having the same issue.

We are checking your issue with the engineering team and will let you know the update.

viktor-shmigol commented 1 month ago

got it

sayali-MSFT commented 1 month ago

@viktor-shmigol - Engineering team also not able to repro for this and Outgoing webhook is working as expected.

Could you please follow this sample- Microsoft-Teams-Samples/samples/outgoing-webhook at main · OfficeDev/Microsoft-Teams-Samples (github.com)

viktor-shmigol commented 1 month ago

@sayali-MSFT Sorry, but I've tested outgoing webhooks with that sample multiple times. I told about it. It also receives double posts!. Let me know If you need to provide with more details.

sayali-MSFT commented 1 month ago

@viktor-shmigol -Can we get network capture when the user at mentions the Outgoing webhook bot in Teams?

viktor-shmigol commented 1 month ago

@sayali-MSFT Here is the messages payload:


{
  "id": "-1",
  "type": "Message",
  "conversationid": "19:VIK4fAKZBU8w1fop1aC7mCT6lPXXLZzY0kOA9-9aFBo1@thread.tacv2",
  "conversationLink": "blah/19:VIK4fAKZBU8w1fop1aC7mCT6lPXXLZzY0kOA9-9aFBo1@thread.tacv2",
  "from": "8:orgid:55ffdb24-452a-43c8-b7d2-da6e2693c956",
  "composetime": "2024-09-13T12:17:16.503Z",
  "originalarrivaltime": "2024-09-13T12:17:16.503Z",
  "content": "<p><readonly class=\"skipProofing\" contenteditable=\"false\" spellcheck=\"false\" itemtype=\"http://schema.skype.com/Mention\"><span itemtype=\"http://schema.skype.com/Mention\" itemscope itemid=\"0\">Test</span></readonly>&nbsp;<readonly class=\"skipProofing\" contenteditable=\"false\" spellcheck=\"false\" itemtype=\"http://schema.skype.com/Mention\"><span itemtype=\"http://schema.skype.com/Mention\" itemscope itemid=\"2\">webhook</span></readonly>&nbsp;Hello world</p>",
  "messagetype": "RichText/Html",
  "contenttype": "Text",
  "imdisplayname": "Viktor Shmigol",
  "clientmessageid": "7463643863206225656",
  "callId": "",
  "state": 0,
  "version": "0",
  "amsreferences": [],
  "properties": {
    "importance": "",
    "subject": "",
    "title": "",
    "cards": "[]",
    "links": "[]",
    "mentions": "[{\"@type\":\"http://schema.skype.com/Mention\",\"itemid\":\"1\",\"mri\":\"28:22e50a9b-80cc-4eab-a092-ce64796d28d7\",\"mentionType\":\"webhook\",\"displayName\":\"Test\"},{\"@type\":\"http://schema.skype.com/Mention\",\"itemid\":\"3\",\"mri\":\"28:22e50a9b-80cc-4eab-a092-ce64796d28d7\",\"mentionType\":\"webhook\",\"displayName\":\"webhook\"}]",
    "onbehalfof": "[{\"itemid\":\"0\",\"mri\":\"00f578f9-393a-4390-a470-5dfb8a3acab4\",\"mentionType\":\"webhook\",\"displayName\":\"Test\"},{\"itemid\":\"2\",\"mri\":\"00f578f9-393a-4390-a470-5dfb8a3acab4\",\"mentionType\":\"webhook\",\"displayName\":\"webhook\"}]",
    "files": "[]",
    "policyViolation": null
  },
  "postType": "Standard",
  "crossPostChannels": []
}

sayali-MSFT commented 1 month ago

@viktor-shmigol -Thanks for sharing the details. We will provide this to the engineering team and let you know the updates.

sayali-MSFT commented 2 weeks ago

@viktor-shmigol - The engineering team is also unable to reproduce the issue. For further investigation, they need a network or HAR capture from your end, as this will provide the necessary headers to trace the logs effectively. Please generate the HAR file during the reproduction of the issue and share it with us. This will help us identify the root cause more efficiently.

Thank you!

MicrosoftDocs / msteams-docs