Encoding::CompatibilityError

[hughkolias]

Has anyone else run into a similar issue when parsing the email body from SNS?

Encoding::CompatibilityError: incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)

I realize it’s erroring in the griddler gem, but it seems to be how it’s passed from the SNS adapter.

I know @ccallebs has a fork and seems to be actively maintaining the new fork.

Here’s the body of the SNS Request:

redacted

[ccallebs]

@hughkolias I will look into this in the evening, thanks!

[hughkolias]

Thanks @ccallebs - let me know if there is anything on our end that could help.

[hughkolias]

@ccallebs - as per https://github.com/thoughtbot/griddler/issues/304, if we can force encoding on the returned body it should solve the issue.

I can submit a PR for it if you'd like?

Long story short is that the body that's being presented to Griddler to parse is encoded in US-ASCII (eg ASCII-8BIT).

I believe the underlying issue is related to the SES adapter decoding the Base64 encoded payload. decode64 always returns a US-ASCII encoded string (since it has no idea what the original encoding of the string was). The encoding has to be forced to UTF-8 after it is decoded.

You can test this yourself:

require 'base64'

str = "foobar" str.encoding # b64str = Base64.encode64(str) b64str.encoding # Base64.decode64(b64str).encoding # Also, I just want to make sure you're ok with having posted that email body publicly. There appear to be a few email addresses just in the JSON payload. Not sure what content could be in the message body itself.

[ccallebs]

@hughkolias If you'd like to submit a PR that would be great, please just make sure to write an accompanying spec that reproduces the bug. Also, please base the changes off of my fork for the time being.

[hughkolias]

Thanks @ccallebs - Let me know your thoughts: https://github.com/ccallebs/griddler-amazon_ses/pull/3

[hughkolias]

addressed via ccallebs#3