Closed ddhp closed 3 weeks ago
Thank you for your question. Firstly, for signature verification, it is necessary to use the body as it is. Therefore, the webhook recipient don't have to rewrite body.
However, It is possible that Twilio Serverless or JSON.stringify is unintentionally changing characters, and you might need to specifically escape emojis to revert this.
Please ensure that the received webhook, including spaces, has not been altered.
Hi @eucyt, thanks for your reply!
We didn't use Twilio Serverless, we directly use JSON.stringify
to parse JSON payload into string. I've looked up and JSON.stringify
wouldn't escape emojis, e.g JSON.stringify({a: '🤨'}) // '{"a":"🤨"}'
Code below is from LINE dev documents, and if we tried body
as {text: JSON.stringify("🤨")}
, the calculated signature wouldn't match the one presented in request header
const crypto = require("crypto");
const channelSecret = "..."; // Channel secret string
const body = JSON.stringify("{ text: 🤨}"); // Request body string
const signature = crypto
.createHmac("SHA256", channelSecret)
.update(body)
.digest("base64");
// Compare x-line-signature request header and the signature
but if we escaped the emojis, i.e make the body as JSON.stringify({text: "\uD83E\uDD28"})
, it matches the signature in request header. (message api received 🤨
and signature is calculated as \uD83E\uDD28
)
Does this mean LINE message API would escape some unicodes to string? Or are we missing something else?
Thanks for your reply in advance!
It is likely that this issue is caused not by the escaping of emojis, but by JSON.stringify
removing spaces or changing some charactors. JSON.stringify
converts {a: '🤨'}
to {"a":"🤨"}
(with spaces removed and so on). This diff from the request body cause a signature verification failure.
Could you please verify the signature without using JSON.stringify, as described in this document?
const crypto = require("crypto");
const channelSecret = "..."; // Channel secret string
const body = "..."; // Request body string
const signature = crypto
.createHmac("SHA256", channelSecret)
.update(body)
.digest("base64");
// Compare x-line-signature request header and the signature
in the example, .update
only takes type string or NodeJS.ArrayBufferView
, so we have to make JSON body into a string by JSON.stringify
code below is with a real payload body from message API webhook
const receivedBody = {"destination":"uce28b96d590a32aca613ed9401d290fa","events":[{"type":"message","message":{"type":"text","id":"521148777382543885","quoteToken":"qj_8_gD5-JSjuOiDF4tBP0ijL2m0jMUNNXbKIcKY2P7fTLAE0qMAUhuP0z8UiWh5OKZMlB5g5uPUou8lD9Wbotsb60KfiNO6ZHSzIYIAb2BQzKsmK8ZQPM0zjuBZ2ystszRajBVXGqRyCWi6pKlUSg","text":"🤨"},"webhookEventId":"01J533PQSZMT61R0H1W2X3Q73Z","deliveryContext":{"isRedelivery":false},"timestamp":1723460181639,"source":{"type":"user","userId":"u6afc4dac41d4a41969852de5d4b28a0c"},"replyToken":"53442902dafb45f18a7962ecb1b2daab","mode":"active"}]}
const bodyInString = JSON.stringify(receivedBody)
// '{"destination":"uce28b96d590a32aca613ed9401d290fa","events":[{"type":"message","message":{"type":"text","id":"521148777382543885","quoteToken":"qj_8_gD5-JSjuOiDF4tBP0ijL2m0jMUNNXbKIcKY2P7fTLAE0qMAUhuP0z8UiWh5OKZMlB5g5uPUou8lD9Wbotsb60KfiNO6ZHSzIYIAb2BQzKsmK8ZQPM0zjuBZ2ystszRajBVXGqRyCWi6pKlUSg","text":"🤨"},"webhookEventId":"01J533PQSZMT61R0H1W2X3Q73Z","deliveryContext":{"isRedelivery":false},"timestamp":1723460181639,"source":{"type":"user","userId":"u6afc4dac41d4a41969852de5d4b28a0c"},"replyToken":"53442902dafb45f18a7962ecb1b2daab","mode":"active"}]}'
original JSON body doesn't have any space between key and value and there is no space after JSON.stringify
as well. AFAICS JSON.stringify
is doing as expected.
Is there anything else we should check?
Have you try sending that emoji to any OA? We tried it on other OA and we can reproduce it as well(no reply from OA)
Thank you
I'm sorry for the delayed response and any confusion caused.
Let's organize our thoughts.
The main reasons are to verify the sender and to ensure that the body has not been tampered with. In other words, you should verify the raw body received.
The root cause of the current error is that the received body has already been deserialized into a dictionary, and you are trying to re-serialize it into a string by JSON.stringify
. This cannot be considered a raw body and is not suitable for verification.
For example,🤨 and ✋✊ were correctly verified on a simple Node server like the one below. This indicates that the type of emoji is not the root cause of the verification failure.
const http = require("http");
const crypto = require("crypto");
const server = http.createServer((req, res) => {
const channelSecret = "..."; // Channel secret string
let body = "";
req.on("data", (chunk) => {
body += chunk.toString();
});
req.on("end", () => {
// Signature verification
const signature = crypto
.createHmac("SHA256", channelSecret)
.update(body)
.digest("base64");
console.log(signature == req.headers["x-line-signature"]);
res.statusCode = 204;
res.end();
});
});
const port = 3000;
const hostname = "127.0.0.1";
server.listen(port, hostname, () => {
console.log(`Server running at http://${hostname}:${port}/`);
});
To reiterate, the solution is to use the raw body as it is for verification. I am not sure how to do this with Twilio, but if you cannot obtain the raw request body, you will not be able to verify any webhook, not just LINE. We recommend using frameworks that can correctly obtain the raw body, such as Express, Hono, or a simple Node server.
Thank you for using LINE's Webhook. I hope your issue gets resolved.
Thanks for your reply! We solved it!
The problem is from express.json
which is a middleware parse payload body to JSON.
When user send 🤨
, LINE message API would send escaped string \uD83E\uDD28
as body to webhook endpoint.
After it's processed by express.json
, it will be encoded to 🤨, so signature calculated from the json body after JSON.stringify
wouldn't match.
Use express.raw to parse payload body will solve this.
Thanks again
Bug Report
Describe the bug A similar issue had been filed in the php sdk repo a few years ago.
We tried the solution concluded from the issue. Basically we can see LINE message API would escape emoji to unicode string inside the message body, e.g 🤨 ->
\uD83E\uDD28
before calculating signature, so we also modify the message body the same way.But this solution doesn't work for all the emojis. AFAIK, ✋✊ won't be escaped by LINE message api.
So in order for our apps to validate signatures properly, can LINE provide us the rule of escaping emoji and the range of unicodes would be escaped?
Thank you
To Reproduce follow signature validation from the official guide and send some emojis to the OA.
Expected behavior Able to validate signature from message api webhook
Screenshots N/A
Environment (please complete the following information): N/A
Additional context N/A