Unicode error on email send

[hjfreyer]

Python sucks... @brad could you take a look at this?

'ascii' codec can't encode character u'\xe9' in position 13: ordinal not in range(128) Traceback (most recent call last): File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in call rv = self.handle_exception(request, response, e) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in call rv = self.router.dispatch(request, response) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher return route.handler_adapter(request, response) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in call return handler.dispatch() File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch return self.handle_exception(e, self.app.debug) File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch return method(_args, _kwargs) File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 309, in post self.run_from_request() File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 304, in run_from_request run(self.request.body) File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/deferred/deferred.py", line 146, in run return func(_args, _kwds) File "/base/data/home/apps/s~mayday-pac/1.375714989356860479/main.py", line 43, in send_thank_you message.body = open('email/thank-you.txt').read().format(**format_kwargs) UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 13: ordinal not in range(128)

[hjfreyer]

For more background, there seems to be only one such email. I imagine the pledger had an accent in his name.

[jtolio]

lol, one time i was working on this system for the ads team and it was supposed to send error email to dss-alerts@google.com, but what happened is the email sending method expected an iterable of email addresses, so it got "d", "s", "s", etc. i was supposed to take care of all the unicode testing, and i wasn't quite done by our first deployment. we deployed and rob pike calls the tech lead: "why are you spamming my email address with unicode errors?! i got 700 this morning, is this a joke?". hilarious. everyone should have one letter email addresses.

unfortunately, it doesn't look like this issue is actually fixed: https://code.google.com/p/googleappengine/issues/detail?id=639

we can either convert name (which is the unicode type) to a utf8-encoded string type and hope for the best, or we can strip out non-ascii runes out of the unicode object

name.encode('utf8') will return a string instead of a unicode object so it's not trying to encode any characters with the ascii codec. "".join((b for b in name.encode('utf8') if ord(b) < 128)) will strip out non-ascii runes, but there's i'm sure a better way

[brad]

Correction, Python 2.x sucks ;-)

[jtolio]

lol @brad

[jtolio]

these guys claim we should use the python 2.5 app engine? could that possibly be true still? i don't believe it is

http://stackoverflow.com/questions/5066027/unicodedecodeerror-when-sending-mail-using-appengine

[hjfreyer]

No, that's deprecated hard, I think.

On Friday, May 9, 2014 11:58:34 AM, JT notifications@github.com wrote:

these guys claim we should use the python 2.5 app engine? could that possibly be true still? i don't believe it is

http://stackoverflow.com/questions/5066027/unicodedecodeerror-when-sending-mail-using-appengine

— Reply to this email directly or view it on GitHubhttps://github.com/MayOneUS/app-engine/issues/39#issuecomment-42682226 .

[jtolio]

oh ho! TIL

name. encode("ascii", errors="ignore")

[hjfreyer]

Ew. Ewewew. Okay

On Friday, May 9, 2014 12:02:42 PM, JT notifications@github.com wrote:

oh ho! TIL

name. encode("ascii", errors="ignore")

— Reply to this email directly or view it on GitHubhttps://github.com/MayOneUS/app-engine/issues/39#issuecomment-42682709 .

[brad]

LOL @jtolds assigning to you

[jtolio]

lol sorry. ideally there's some way of telling the email message to use a utf8 content-encoding, but i don't see one. do either of you?

[jtolio]

lolol this is the worst http://stackoverflow.com/a/15640761/379568, like, a quoted-printable injection attack to get utf8? :(

[hjfreyer]

Yeah, a hack is better than preventing this guy from getting his receipt.

Maybe next time I will go with go... :P

[hjfreyer]

Pushed and triggered the email send. Went though (hopefully not looking like ass).

On Fri May 09 2014 at 12:16:01 PM, JT notifications@github.com wrote:

Closed #39 https://github.com/MayOneUS/app-engine/issues/39.

— Reply to this email directly or view it on GitHubhttps://github.com/MayOneUS/app-engine/issues/39#event-119608715 .