guilhemmarchand / TA-ms-teams-alert-action

This application provides alert actions for Microsoft Teams messages publication to allows advanced messages publication from Splunk.
7 stars 5 forks source link

duplicate messages are posting #32

Closed gowtham495 closed 3 years ago

gowtham495 commented 4 years ago

Add-On was working good so far. But suddenly one alert triggered duplicate messages to Teams Channel and then stopped triggering altogether.

guilhemmarchand commented 4 years ago

Hi @gowtham495

Do logs show any interesting?

This looks more to me like an alert / Splunk issue question, to avoid generating duplicated messages you need to rely on a proper thorttling in the alert config.

I would advise you to have a simple alert generating one result with makeresults for example, and test that you are still able to generate a message.

gowtham495 commented 4 years ago

Actually it was working fine for past 30 days or so. When i check internal logs, only one event registered for alert trigger but i could see 3 same messages in Teams. So i can confirm this is not Splunk issue.

On Wed, 7 Oct, 2020, 9:04 pm Guilhem Marchand, notifications@github.com wrote:

Hi @gowtham495 https://github.com/gowtham495

Do logs show any interesting?

This looks more to me like an alert / Splunk issue question, to avoid generating duplicated messages you need to rely on a proper thorttling in the alert config.

I would advise you to have a simple alert generating one result with makeresults for example, and test that you are still able to generate a message.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/guilhemmarchand/TA-ms-teams-alert-action/issues/32#issuecomment-705018030, or unsubscribe https://github.com/notifications/unsubscribe-auth/AILQPWHGX2NFVMCF4PDG5ETSJSDBJANCNFSM4SHPVANA .

guilhemmarchand commented 4 years ago

Can you describe more clearly your problem please. Are we talking about duplicate messages, or messages that are not being created anymore?

The UI provides you with all the shortcuts to the alert logs and analytics, please have a look.

guilhemmarchand commented 4 years ago

If you want help with anything you need to be at the minimal providing information, context, logs, screen shots, investigations, whatever. Just saying something does not work will not help anyone.

Every call is recorded in:

(index="_internal" OR index="cim_modactions") (source="*ms_teams_publish_to_channel_modalert.log")

Check the UI too, perform testing.

Checkout the resilient store logs as well:

(index="_internal" OR index="cim_modactions") (source="*ms_teams_publish_to_channel_replay_modalert.log")

Potentially I could think about an issue on this side which would have generated duplicates, maybe MS was having troubles, maybe you were, but that shall not have been the case.

I understand you had dup messages but I am not sure to understand if the addon continues to generate messages when you are expecting them or not.

guilhemmarchand commented 4 years ago

Get me some traces @gowtham495 and I will be more than happy to help and investigate ;-)

gowtham495 commented 4 years ago

Apologies. I'm working in client machine, so neither I have access to the source you mentioned or take snapshot of the same. But I'm using version 1.0.12 and i believe I'm having duplication message issue as mentioned in the release notes. Is there a way to resolve this ?

[image: image.png]

On Wed, 7 Oct 2020 at 22:38, Guilhem Marchand notifications@github.com wrote:

Get me some traces @gowtham495 https://github.com/gowtham495 and I will be more than happy to help and investigate ;-)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/guilhemmarchand/TA-ms-teams-alert-action/issues/32#issuecomment-705073323, or unsubscribe https://github.com/notifications/unsubscribe-auth/AILQPWGZUFOUERB7OJZXNZDSJSOBJANCNFSM4SHPVANA .

guilhemmarchand commented 4 years ago

@gowtham495 Right I see.

Ok does it happens often / regurlalry or it happened once only?

I believe you may have faces some unexpected dup creation due to temporary issues on MS side or Network latency, it's not supposed to be happening but it's not fully impossible either.

I will have a look as I believe I could add an additional safety layer, however please answer to the question and try to keep the app up to date.

gowtham495 commented 4 years ago

It happened once, then we disabled that alert to avoid spamming the channel.

Then we updated the Add on to latest version.

Now, Splunk is running the alert and it also gets added to triggered alert. But no message is getting posted in Teams.

Message is getting lost somewhere in the middle.

On Thu, 8 Oct, 2020, 6:05 pm Guilhem Marchand, notifications@github.com wrote:

@gowtham495 https://github.com/gowtham495 Right I see.

Ok does it happens often / regurlalry or it happened once only?

I believe you may have faces some unexpected dup creation due to temporary issues on MS side or Network latency, it's not supposed to be happening but it's not fully impossible either.

I will have a look as I believe I could add an additional safety layer, however please answer to the question and try to keep the app up to date.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/guilhemmarchand/TA-ms-teams-alert-action/issues/32#issuecomment-705538315, or unsubscribe https://github.com/notifications/unsubscribe-auth/AILQPWAUJIYMT4JF7J2P5GLSJWW2XANCNFSM4SHPVANA .

guilhemmarchand commented 4 years ago

When the alert triggers does it generate activity in the MS Addon logs? (in the UI or the logs searches I shared earlier)

This is only right way to know if the action is fired by Splunk or not.

If the action cannot be fired due to a technical failure, this will be in the logged. If there are no traces, then the action is not getting fired.

Please test with a brand new alert that you create for the testing purpose, to see if your issue is happening widely or just in the scope of this alert

| makeresults
| eval message="This is a testing message, please ignore."

Create an alert on top of that, add the alert action and check.

Simple easy peasy.

guilhemmarchand commented 3 years ago

Closing this issue, if anything new feel free to re-open.