search

emencia / emencia-django-newsletter

An app for sending newsletter by email to a contact list.
189 stars 72 forks source link

Sending utf-8 e-mail #3

Closed curaloucura closed 14 years ago

curaloucura commented 14 years ago

Hello, I'm having an issue when trying to send newsletter with utf-8, it raises the following error

Traceback (most recent call last): File "./manage.py", line 11, in execute_manager(settings) File "/home/anderson/.virtualenvs/svbcwb/lib/python2.6/site-packages/django/core/management/init.py", line 362, in execute_manager utility.execute() File "/home/anderson/.virtualenvs/svbcwb/lib/python2.6/site-packages/django/core/management/init.py", line 303, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/anderson/.virtualenvs/svbcwb/lib/python2.6/site-packages/django/core/management/base.py", line 195, in run_from_argv self.execute(_args, _options.dict) File "/home/anderson/.virtualenvs/svbcwb/lib/python2.6/site-packages/django/core/management/base.py", line 222, in execute output = self.handle(_args, _options) File "/home/anderson/.virtualenvs/svbcwb/lib/python2.6/site-packages/django/core/management/base.py", line 351, in handle return self.handle_noargs(options) File "/home/anderson/.virtualenvs/svbcwb/lib/python2.6/site-packages/emencia/django/newsletter/management/commands/send_newsletter.py", line 19, in handle_noargs mailer.run() File "/home/anderson/.virtualenvs/svbcwb/lib/python2.6/site-packages/emencia/django/newsletter/mailer.py", line 47, in run message = self.build_message(contact) File "/home/anderson/.virtualenvs/svbcwb/lib/python2.6/site-packages/emencia/django/newsletter/mailer.py", line 77, in build_message message.attach(MIMEText(content_text, 'plain', 'UTF-8')) File "/usr/lib/python2.6/email/mime/text.py", line 30, in init** self.set_payload(_text, _charset) File "/usr/lib/python2.6/email/message.py", line 224, in set_payload self.set_charset(charset) File "/usr/lib/python2.6/email/message.py", line 266, in set_charset self._payload = charset.body_encode(self._payload) File "/usr/lib/python2.6/email/charset.py", line 387, in body_encode return email.base64mime.body_encode(s) File "/usr/lib/python2.6/email/base64mime.py", line 147, in encode enc = b2a_base64(s[i:i + max_unencoded]) UnicodeEncodeError: 'ascii' codec can't encode characters in position 29-30: ordinal not in range(128)

But when I change these two lines on mailer.py

message.attach(MIMEText(content_text.encode('utf-8'), 'plain', 'UTF-8')) message.attach(MIMEText(content_html.encode('utf-8'), 'html', 'UTF-8'))

It sends fine, but the text gets wrong encoding. I'm using SQLite3 so I don't think it should be a problem of table encoding and when the content don't have any utf-8 character but the template has, it also raises that error, so it's not database encoding for sure.

curaloucura commented 14 years ago

Oh yeah, and I checked the file encoding and the browser encoding and both are utf-8

curaloucura commented 14 years ago

Hmm, weird enough, it was just on my local computer, when I uploaded to my server it was sent fine. maybe some version conflicts or hidden settings. But still the encode was needed, when not encoding I get the first error too.

Fantomas42 commented 14 years ago

Ok, I will check this kind of errors, but are you sure that you don't have any accentuated characters in your newsletter content ? So for your templates I recommended you to use HTML characters, like é

I will try to reproduce this bug, so can you give me more informations or more data ?

curaloucura commented 14 years ago

So, I tried with non ASCII data on both, just template and just database and on both situations the conversion was required. If I only use ASCII data it works fine. At first I thought about using html entities but I realized that if I have to send a newsletter on japanese (I might have this need soon) I will have a problem. For instance I send e-mails with UTF-8 just using django and it works great. Maybe there's something wrong with Beautiful Soup encoding results?

Fantomas42 commented 14 years ago

Ok I'm testing with non ASCII data, and using GMAIL as the SMTP, and the sending work fine, but the accentued characters or Japanese character are doesn't displayed in my mail client.

I'm working on it

Fantomas42 commented 14 years ago

Hum, this bug is very weird because, if a send a mail in test, it's works fine, but with the command, it raise this error.

curaloucura commented 14 years ago

hmm, I will try to do some tests tonight at home and see if I can get any other clue. thanks!

Fantomas42 commented 14 years ago

Ok after multiple try to reproduce the bug, it has been fixed ! And japanese character are correctly handled. :)

To view the modifications : http://github.com/Fantomas42/emencia-django-newsletter/commit/0dd707e12123e9c7612d581839777269ffb44a15

Thanx for the bug !