Webhook validation fails

[andrii2020]

Webhook validation fails

Hi! I'm trying to add validation to the webhook. When I'm trying to use a key, timestamp, a payload from your GitHub file with tests everything works fine. But when I generate my own key and try to use the request body which I catch via webhook.site it fails. I can provide body, key, headers which I use. I have the last version of SendGrid installed. Could you please check if newly generated keys works fine for you. Thanks!

Steps to Reproduce

1. Enable webhook in the admin panel
2. Enable validation
3. Add webhook.site link and send test request
4. Substitute received request and send it to my webhook
5. Receive fail because of bad key

[thinkingserious]

Hello @andrii2020,

One item to check is to make sure you are using the payload in raw bytes form.

Yes, providing sample data that we can use to reproduce would be helpful.

Thank you!

With best regards,

Elmer

[andrii2020]

Verification Key: MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEpWnjbw+pEuD8lTRd0aLycMT6AYQfx1GqxXQkDQtb0UsncIxsDxCRl9LYYv/p8dwZuEww7nPpUvR+F8W6ZItXjw==

x-twilio-email-event-webhook-signature MEQCIHQkjRdbcbM/gXxai4eTHRDxsa6XyiIfXrqD6tBV7FUDAiAPZQSdHQU9O4Q/U9xA1wiDb8SppEgQbilr/Sb7sI5zCQ==

x-twilio-email-event-webhook-timestamp 1604746997

[{"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"processed","category":["cat facts"],"sg_event_id":"Pi4uHYgstmIdRi-ZdO_ggg==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"deferred","category":["cat facts"],"sg_event_id":"bdD_tns7-PP42T7wi8JjBA==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0","response":"400 try again later","attempt":"5"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"delivered","category":["cat facts"],"sg_event_id":"RMmLFTkThVMMNSTUwe7ONA==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0","response":"250 OK"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"open","category":["cat facts"],"sg_event_id":"MBG5Xzl4mFtYaNnsffsZZA==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0","useragent":"Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)","ip":"255.255.255.255"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"click","category":["cat facts"],"sg_event_id":"Flzg2OEW2UQIzsW1bSBNSA==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0","useragent":"Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)","ip":"255.255.255.255","url":"http://www.sendgrid.com/"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"bounce","category":["cat facts"],"sg_event_id":"1IHdcZbI0LDWGHEw4kTSAA==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0","reason":"500 unknown recipient","status":"5.0.0"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"dropped","category":["cat facts"],"sg_event_id":"bgV-x0kIOFi1Koufqs72cA==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0","reason":"Bounced Address","status":"5.0.0"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"spamreport","category":["cat facts"],"sg_event_id":"NBuZRGBjepr_LQFooZxC7g==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"unsubscribe","category":["cat facts"],"sg_event_id":"kiEES_suFldvMlepvYA7Xg==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0"}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"group_unsubscribe","category":["cat facts"],"sg_event_id":"wNf0IuGAVoAsmihEU-FBHw==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0","useragent":"Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)","ip":"255.255.255.255","url":"http://www.sendgrid.com/","asm_group_id":10}, {"email":"example@test.com","timestamp":1604746997,"smtp-id":"\u003c14c5d75ce93.dfd.64b469@ismtpd-555\u003e","event":"group_resubscribe","category":["cat facts"],"sg_event_id":"dxIWjlfzpye4b3zqaKSuKw==","sg_message_id":"14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0","useragent":"Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)","ip":"255.255.255.255","url":"http://www.sendgrid.com/","asm_group_id":10}]

[thinkingserious]

Another thing to check is that you are including a trailing carriage return and newline.

[andrii2020]

Yeah, everything works fine with data from your tests. but when I substitute data which I shared previously it fails.

[eshanholtz]

I'm able to reproduce the failure you're seeing with the data you shared. This issue has been added to our internal backlog to be prioritized. Pull requests and +1s on the issue summary will help it move up the backlog.

[fmaillet24]

+1 exactly the same error with exactly the same setup. It works with your test file, but not with webhook.site

[AntovskiAntonio]

+1 I have the exact same problem. I've also tried to add the trailing carriage return and newline, as suggested by @thinkingserious, but that didn't help either. Any suggestion on how to make this work?

[nawar]

Any resolution on this?

[shwetha-manvinkurke]

Have you all tried with a trailing \r\n after every event? Here is an example in node

[nawar]

@shwetha-manvinkurke yes we did in our case. The test cases in the repo work fine without issues, it seems something has changed from SendGrid side.

[marine-mb]

+1 verify_signature is not accepting raw bytes directly as specified in example (type for payload is a string and is concatenated with timestamp). I tried with decode('latin-1') as suggested in https://github.com/sendgrid/sendgrid-python/pull/990/files and I added the trailing carriage return and newline. However, the check is still failing.

Any updates on this issue ? :)

[corno93]

+1 I'm confused too I've written a test that uses data from yours here. The test passes.

When I use my signed public verification key with integration test data triggered from sendgrid's event webhook UI it fails

Any idea how to resolve this?

[corno93]

I got it working by using the raw request body and decoding using latin-1. I didnt have to use the carriage return and new line. I also deleted and got a new API key and web hook verification key (not sure if that helps but it may)

Here the request object is from drf

        eventwebhook = EventWebhook(settings.SENDGRID_PUBLIC_WEBHOOK_KEY)
        is_valid = eventwebhook.verify_signature(
            request.body.decode('latin-1'),
            request.headers[EventWebhookHeader.SIGNATURE],
            request.headers[EventWebhookHeader.TIMESTAMP],
        )

[marine-mb]

@corno93 I tried you method but it still doesn't work for me... 😔

@thinkingserious @eshanholtz Any updates on this issue ? 😊

[HendJordan]

+1

EDIT: I was just able to get this working with the following modifications to the payload:

# strip whitespace {"key": "value", "foo": bar} -> {"key":"value","foo":bar}
data = json.dumps(json.loads(request.get_data(as_text=True)), separators=(",", ":"))
# requires newline and carriage return for each object in list + one trailing at the end
payload = "},\r\n{".join(data.split("},{")) + "\r\n"

[marceloaba]

+1 Apparently multiple people are having the same issue with the signature validation. Is there any expected date to fix this issue?

[ranjanp75]

I confirm that @corno93's solution works by using raw request body and decoding to "latin-1" alone. Thanks a lot

[dtiesling]

UPDATE: The issue was a bit of magic in our requests I was unaware was happening. There was middleware doing some html escaping in the payloads. After figuring out how to retrieve the truly raw payload this is working as expected.

If you are having issues it's worth watching the network requests upstream of python to make sure nothing is happening to the payloads.

+1 Having the same issue and tried all of the workarounds from above. I'm attempting to handle the webhooks in a Flask view with sendgrid==6.11.0.

Does anyone have a working code snippet that formats the payload from a Flask request? Or am I missing something else in the code below?

Latest code snippet testing the latin-1 decoding workaround

EventWebhook(public_key=public_key).verify_signature(
        payload=request.data.decode("latin-1"),
        signature=request.headers.get("X-Twilio-Email-Event-Webhook-Signature"),
        timestamp=request.headers.get("X-Twilio-Email-Event-Webhook-Timestamp"),
    )

Example payload

verification key: MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE5Qn2Y4zzFuRZGgCZagcofv4gE2mU9BtV18BO0kUVS/6lwayLXJfEoNEseO5auXrPS2ds/qpCezn1Z35qB6DBSg==

signature: MEYCIQDuyIVG3XI1+Pe/Sq/Nr6tgSEYxOS61RCKFNHA3jsfedwIhAMFT97AX4eWDh7GJabWwVqQvZv6pcM3cypz83rbQh4tx

timestamp: 1715877013

payload: [{"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "processed", "category": ["cat facts"], "sg_event_id": "3JMizkAyFQ2KGjExu_FXrQ==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "deferred", "category": ["cat facts"], "sg_event_id": "sF4uHYvI3Q6RYzIz94oagw==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0", "response": "400 try again later", "attempt": "5"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "delivered", "category": ["cat facts"], "sg_event_id": "6jZqpVNnfIu4Jio1pgDWKg==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0", "response": "250 OK"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "open", "category": ["cat facts"], "sg_event_id": "sTyTPHSu6yb9wT4Uguozmg==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0", "useragent": "Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)", "ip": "255.255.255.255"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "click", "category": ["cat facts"], "sg_event_id": "IgBBSc53jusTlIZbRMHcYQ==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0", "useragent": "Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)", "ip": "255.255.255.255", "url": "http://www.sendgrid.com/"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "bounce", "category": ["cat facts"], "sg_event_id": "UfnOu16YiTUbkTzFt2AXIA==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0", "reason": "500 unknown recipient", "status": "5.0.0"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "dropped", "category": ["cat facts"], "sg_event_id": "qjEbaeJnpVP-SWjXo1Sehw==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0", "reason": "Bounced Address", "status": "5.0.0"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "spamreport", "category": ["cat facts"], "sg_event_id": "Ww4XLyM17dl9Y5ZI9g2y4w==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "unsubscribe", "category": ["cat facts"], "sg_event_id": "lpnaz0P8OrjAu3EXZxs3Vg==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0"}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "group_unsubscribe", "category": ["cat facts"], "sg_event_id": "9aCVpaBszvmQ7gpk5LcD4w==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0", "useragent": "Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)", "ip": "255.255.255.255", "url": "http://www.sendgrid.com/", "asm_group_id": 10}, {"email": "example@test.com", "timestamp": 1715877013, "smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>", "event": "group_resubscribe", "category": ["cat facts"], "sg_event_id": "IGR4PljOZkODcwc47CGZrg==", "sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0", "useragent": "Mozilla/4.0 (compatible; MSIE 6.1; Windows XP; .NET CLR 1.1.4322; .NET CLR 2.0.50727)", "ip": "255.255.255.255", "url": "http://www.sendgrid.com/", "asm_group_id": 10}]