Using webhook notification provider with multiline messages results in broken payloads

[lordmofisto]

⚠️ Please verify that this bug has NOT been raised before.

• [X] I checked and didn't find similar issue

🛡️ Security Policy

• [X] I agree to have read this project Security Policy

📝 Describe your problem

I recently updated to 1.23.1.

I am using a webhook notification to send alerts when monitored services are down using PING

I can do a test send and I receive it. I also receive notifications when a service comes back up after being down, but the alert letting me know a service went down in the first place isn't coming through. The logs are included below

📝 Error Message(s) or Log

2023-09-20T20:51:51.094326466Z 2023-09-20T14:51:51-06:00 [MONITOR] ERROR: Cannot send notification to Uptime-Kuma Dectection
2023-09-20T20:51:51.096885926Z Error: Error: AxiosError: Request failed with status code 400 {"status":400,"error":{"message":"Unexpected token \""}}
2023-09-20T20:51:51.096922873Z     at Webhook.throwGeneralAxiosError (/app/server/notification-providers/notification-provider.js:38:15)
2023-09-20T20:51:51.096953290Z     at Webhook.send (/app/server/notification-providers/webhook.js:57:18)
2023-09-20T20:51:51.096960747Z     at runMicrotasks (<anonymous>)
2023-09-20T20:51:51.096967522Z     at processTicksAndRejections (node:internal/process/task_queues:96:5)
2023-09-20T20:51:51.096973864Z     at async Function.sendNotification (/app/server/model/monitor.js:1380:21)
2023-09-20T20:51:51.096980236Z     at async beat (/app/server/model/monitor.js:917:21)
2023-09-20T20:51:51.096988676Z     at async Timeout.safeBeat [as _onTimeout] (/app/server/model/monitor.js:985:17)

🐻 Uptime-Kuma Version

1.23.1

💻 Operating System and Arch

Ubuntu Server 20.04

🌐 Browser

Chrome

🐋 Docker Version

No response

🟩 NodeJS Version

No response

[CommanderStorm]

From the log, it seems like the site you are webhooking to is returning 400 with {"message":"Unexpected token ""}}

• What is your template?
  • If the template is at fault: to check how you need to tweak the template, a interceptor like https://webhook.site/ might be helpfull.
• what service are you monitoring
• Could you have a look at https://github.com/louislam/uptime-kuma/pull/3636 if this maybe clears up some things?

[lordmofisto]

I am using the following

1. Notification Type: Webhook
2. Request Body: Custom Body
3. Custom JSON body (see below)
4. Additional Headers: On/True
5. Additional Header Content (see below)

Custom Body

```json { "attachments": [ { "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "type": "AdaptiveCard", "version": "1.3", "body": [ { "type": "Container", "items": [ { "type": "ColumnSet", "columns": [ { "type": "Column", "width": "stretch", "items": [ { "type": "TextBlock", "text": "A monitored service is down", "weight": "bolder", "size": "large", "color": "warning", "wrap": true }, { "type": "Image", "url": "MYPICURL", "size": "small", "style": "person" } ] } ] } ] }, { "type": "Container", "items": [ { "type": "FactSet", "facts": [ { "title": "Name", "value": "{{ monitorJSON['name'] }}" }, { "title": "Description", "value": "{{ monitorJSON['description'] }}" }, { "title": "Hostname", "value": "{{ monitorJSON['hostname'] }}" }, { "title": "Type", "value": "{{ monitorJSON['type'] }}" } ] } ] }, { "type": "Container", "items": [ { "type": "TextBlock", "text": "{{ msg }}" } ] } ] } ] } ```

Custom Header

```json { "Content-Type": "application/json" } ```

I don't get an invalid token message when I do the test notification or when the service comes back up, so I am not sure why the down notification would be treated differently.

I used the https://webhook.site to test what's being sent and here is what I received.

Monitor is down (raw content):

```json "{ \"attachments\": [ { \"$schema\": \"http://adaptivecards.io/schemas/adaptive-card.json\", \"type\": \"AdaptiveCard\", \"version\": \"1.3\", \"body\": [ { \"type\": \"Container\", \"items\": [ { \"type\": \"ColumnSet\", \"columns\": [ { \"type\": \"Column\", \"width\": \"stretch\", \"items\": [ { \"type\": \"TextBlock\", \"text\": \"A monitored service is down\", \"weight\": \"bolder\", \"size\": \"large\", \"color\": \"warning\", \"wrap\": true }, { \"type\": \"Image\", \"url\": \"MYPICURL\", \"size\": \"small\", \"style\": \"person\" } ] } ] } ] }, { \"type\": \"Container\", \"items\": [ { \"type\": \"FactSet\", \"facts\": [ { \"title\": \"Name\", \"value\": \"Test Ping (Notifications)\" }, { \"title\": \"Description\", \"value\": \"This is a test of the ping to my test container\" }, { \"title\": \"Hostname\", \"value\": \"10.0.3.202\" }, { \"title\": \"Type\", \"value\": \"ping\" } ] } ] }, { \"type\": \"Container\", \"items\": [ { \"type\": \"TextBlock\", \"text\": \"[Test Ping (Notifications)] [🔴 Down] PING 10.0.3.202 (10.0.3.202) 56(84) bytes of data.\n\n--- 10.0.3.202 ping statistics ---\n10 packets transmitted, 0 received, 100% packet loss, time 218ms\n\n\" } ] } ] } ] }" ```

Monitor is back up (raw content)

```json { "attachments": [ { "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "type": "AdaptiveCard", "version": "1.3", "body": [ { "type": "Container", "items": [ { "type": "ColumnSet", "columns": [ { "type": "Column", "width": "stretch", "items": [ { "type": "TextBlock", "text": "A monitored service is down", "weight": "bolder", "size": "large", "color": "warning", "wrap": true }, { "type": "Image", "url": "MYPICURL", "size": "small", "style": "person" } ] } ] } ] }, { "type": "Container", "items": [ { "type": "FactSet", "facts": [ { "title": "Name", "value": "Test Ping (Notifications)" }, { "title": "Description", "value": "This is a test of the ping to my test container" }, { "title": "Hostname", "value": "10.0.3.202" }, { "title": "Type", "value": "ping" } ] } ] }, { "type": "Container", "items": [ { "type": "TextBlock", "text": "[Test Ping (Notifications)] [✅ Up] " } ] } ] } ] } ```

The raw content of the notification sent when the service comes back up is exactly as I have it in the template (and the system variables like msg come through.

[CommanderStorm]

here is the output nicely formatted

```json { "attachments": [ { "$schema": "http://adaptivecards.io/schemas/adaptive-card.json", "type": "AdaptiveCard", "version": "1.3", "body": [ { "type": "Container", "items": [ { "type": "ColumnSet", "columns": [ { "type": "Column", "width": "stretch", "items": [ { "type": "TextBlock", "text": "A monitored service is down", "weight": "bolder", "size": "large", "color": "warning", "wrap": true }, { "type": "Image", "url": "MYPICURL", "size": "small", "style": "person" } ] } ] } ] }, { "type": "Container", "items": [ { "type": "FactSet", "facts": [ { "title": "Name", "value": "{{ monitorJSON['name'] }}" }, { "title": "Description", "value": "{{ monitorJSON['description'] }}" }, { "title": "Hostname", "value": "{{ monitorJSON['hostname'] }}" }, { "title": "Type", "value": "{{ monitorJSON['type'] }}" } ] } ] }, { "type": "Container", "items": [ { "type": "TextBlock", "text": "{{ msg }}" } ] } ] } ] } ```

That does not look faulty, nor like the culprit. Interesting.

I have a few ideas:

• could you test sending this webhook event using postman, curl or equivalent tools to rule out that the template is at fault?
• Where does the token come from? Is it somewhere in the url? How do you authentificate exactly? (what is the notification provider you are using)
• Do you have other monitors running that could produce such a log message (might this log message be misleading?)

[chakflying]

This is quite a nasty bug. Apparently if msg contains newline characters, it causes the whole body to be escaped and enclosed in "", thereby becoming invalid JSON and gets rejected by Teams. Will need to continue to investigate the root cause.

For now, please change this in the template which should mitigate the issue:

            {
              "type": "TextBlock",
              "text": "{{ msg | replace: "\n", "" }}"
            }

[lordmofisto]

That has fixed my issue, thank you very much for your help

[chakflying]

After investigation, this is a combination of 2 things:

• msg containing unescaped newline characters as a footgun:

  When the template engine simply substitutes {{ msg }}, this results in:

      {
        "type": "Container",
        "items": [
          {
            "type": "TextBlock",
            "text": "[Test Bad Ping] [🔴 Down]
  Pinging 107.210.24.15 with 56 bytes of data:
  Request timed out.
  
  Ping statistics for 107.210.24.15:
      Packets: Sent = 1, Received = 0, Lost = 1 (100% loss),
  "
          }
        ]

  Which itself is invalid JSON, but if this got sent the error should be more obvious, if not for

• axios trying to parse the body as JSON again 🤦🏻‍♂️.

  axios would detect that we have set the Content-Type as application/json, and try to parse the data as JSON to trim whitespaces. But since we provided invalid JSON, it then falls back to calling JSON.stringify(data), which created the resulting mess.

I don't see an easy way to fix this. Maybe msg shouldn't contain unescaped newlines? Or maybe we shouldn't even be using axios in this case? In these situations where we want to precisely control over the output, axios is too opinionated to work with.

[DemoJ]

This is quite a nasty bug. Apparently if msg contains newline characters, it causes the whole body to be escaped and enclosed in "", thereby becoming invalid JSON and gets rejected by Teams. Will need to continue to investigate the root cause.这是一个非常讨厌的错误。显然，如果 msg 包含换行符，则会导致整个正文被转义并包含在 "" 中，从而成为无效的 JSON 并被 Teams 拒绝。需要继续调查根本原因。

For now, please change this in the template which should mitigate the issue:现在，请在模板中更改此设置，这应该可以缓解该问题：

            {
              "type": "TextBlock",
              "text": "{{ msg | replace: "\n", "" }}"
            }

After searching for so long, I've finally found the solution. It's a lifesaver!

[JoshTheBlack]

This issue was closed, but should probably remain open until it has been fixed. Can you reopen please?

[CommanderStorm]

maybe we shouldn't even be using axios in this case? In these situations where we want to precisely control over the output, axios is too opinionated to work with

It needs investigating if we can disable the json parsing in axios via a request option. If this is the case, I would suggest to stick with it for the time being. If no such option exists, switching to plain fetch or XMLHttpRequests seems like a good idea.

If a contributor would like to look into this, the webhook notification-provider is located here and the contributing guide (how to setup your environment, ...) is located here