Graph API for Chat not sending notification except for request with validation token.

[damiankucharski]

Steps to reproduce

We are not getting notifications for new teams messages. The MS Graph subscription (chats/getAllMessages?model=B) is active and we also tried renewing it. The validation token is received after the subscription is being created but later no other notification is being received. We did not change anything with our system, the issue appeared today, yesterday everything was working.

Expected behavior

After we send a message in teams, the notification is received as POST request by our API endpoint as usual.

Actual behavior

API does not receive any requests besides the single validation token request: 2024-10-10 09:37:49 INFO got val string request. Source: subscription was requested to made INFO: 52.142.115.31:0 - "POST /graph/notifications?validationToken=Validation%3a+Testing+client+application+reachability+for+subscription+Request-Id%3a+185e6c58-3c20-4adb-ad67-14c878280072 HTTP/1.1" 200 OK

Error details

No response

[MMarciniak103]

Hey, I am colleague from Damian's team. Additional context: we are also using mailFolders('inbox')/messages subscription and this one works correctly. It is only the chats/getAllMessages?model=B that stopped sending notifications

[Prasad-MSFT]

Ensure that the subscription is being renewed before it expires and that the renewal requests are being processed successfully. Also, are you getting any of these errors? https://learn.microsoft.com/en-us/graph/api/chats-getallmessages?view=graph-rest-1.0&tabs=http#errors

[MMarciniak103]

Hey, The subscription is always renewed before it expires (by lifecycle handler passed in "lifecycleNotificationUrl"). As for the https://graph.microsoft.com/v1.0/users/{user_id}/chats/getAllMessages?$top=2 we cannot run it in delegated mode: {'error': {'code': 'PreconditionFailed', 'message': 'Requested API is not supported in delegated context', 'innerError': {'date': '2024-10-10T11:14:35', 'request-id': '8596cae3-91ca-41d4-bcd9-87dfb3fbc213', 'client-request-id': '8596cae3-91ca-41d4-bcd9-87dfb3fbc213'}}}

but we can confirm that this account have valid office365 licence (it was working for couple of months now, it just stopped suddenly today) You can see that we are also using model=B type so there are no payment related errors. I've tried to delete subscription -> create new one and it was successfull (got validation token request but later no notifications are send from Teams to our handler)

[MMarciniak103]

I've also tried running https://graph.microsoft.com/v1.0/chats/{chat_id}/messages?$top=2 and it runs successfully

[MMarciniak103]

It looks like the similar issue as here: https://github.com/MicrosoftDocs/msteams-docs/issues/11558 although we stopped getting any notifications, while this guys was missing some of them.

[knmarcin]

In my project we have same problem. After couple of months, teams channel messages stopped coming.

[knmarcin]

I think it stopped working around on Oct 9, 2024, 20:00 CEST. Then barely promile of exchanged messages came through my endpoint.

[MMarciniak103]

Hey @knmarcin, it aligns with our observations. Since the beginning of this week (around October 7/8), we began to notice a problem such that some messages were not sent to us, until yesterday we stopped getting any.

[knmarcin]

Yes, the last message we processed was yesterday at 10:07 CEST. Since then, no further messages have been received

[damiankucharski]

@Prasad-MSFT it looks like this may be a global issue, see this link: https://learn.microsoft.com/en-us/answers/questions/2101043/frequent-interruptions-in-graph-api-subscriptions

[Prasad-MSFT]

Hi @damiankucharski , @MMarciniak103 , @knmarcin - we reached out to engineering team regarding this issue. Once we have any update from them, we will share in this thread. Thanks!

[laknoll]

I’ve been experiencing sporadic failures where we’ve not been getting notifications for some messages from time to time (see https://github.com/MicrosoftDocs/msteams-docs/issues/11558).

Since yesterday, we’re also seeing a complete failure to get any notifications for our subscriptions. I’ve been posting some details on tenant, channel, message and subscription IDs that we didn’t receive in the other bug report.

[laknoll]

Just to clarify, for us it’s not getAllMessages that fails, but /teams/{id}/channels/{id}/messages.

[Prasad-MSFT]

Update:

We have raised an ICM for this issue and engineering team is looking into it. We will inform you once we have further update. Thanks!

[damiankucharski]

Thanks!

[hanord]

It seems like we're getting notifications for message subscriptions again. Has the issue been fixed, @Prasad-MSFT ?

[bistech-rcampbell]

Still having the same issue over here but for subscriptions to Microsoft Teams transcripts, tested as of this morning and the service is still not working, I imagine it's the same issue as this. Still happening as of this morning.

Same issue as Damian reports

The validation token is received after the subscription is being created but later no other notification is being received. We did not change anything with our system.