search

thoughtbot / griddler

Simplify receiving email in Rails (deprecated)
http://griddler.io/
MIT License
1.38k stars 199 forks source link

UTF-8 encoding works as expected, not desired #49

Closed lagartoflojo closed 11 years ago

lagartoflojo commented 11 years ago

Hi, I will be processing a lot of email written in Spanish, so I need to keep accented characters, etc. The clean_invalid_utf8_bytes uses 'binary' to encode the messages by default, and otherwise falls back to params[:charsets]. The problem is that, as noted in issue #27, I am losing all accented characters (Hernán comes out as Hernn), and not all adapters provide charset information (Mandrill, which we use, doesn't). I'm not sure how to solve this issue... The ideal solution would be to parse the raw email and extract the charsets from there, but that's a pain. Maybe adding a config to set the default encoding, in order to override 'binary'? Thanks!

lagartoflojo commented 11 years ago

So I went ahead and parsed raw email using the Mail gem and then extracted the charsets from there (I applied this on top of the Mandrill adapter):

https://github.com/familink/griddler/commit/9515d4b921325da90033010d23c3daab3eb831df

This allows me to have the real charsets instead of guessing. How does that look?

calebhearth commented 11 years ago

@lagartoflojo go ahead and open a pull request with that branch. @jayroh was the one who was working on this, so I'm going to ask that he look at it when he is back from vacation.

jayroh commented 11 years ago

Let it rip whenever it suits you. I'm back from my trip.

On Saturday, April 20, 2013, Caleb Thompson wrote:

@lagartoflojo https://github.com/lagartoflojo go ahead and open a pull request with that branch. @jayroh https://github.com/jayroh was the one who was working on this, so I'm going to ask that he look at it when he is back from vacation.

— Reply to this email directly or view it on GitHubhttps://github.com/thoughtbot/griddler/issues/49#issuecomment-16706543 .

jayroh commented 11 years ago

@lagartoflojo would you be able to help us by writing a failing test with your desired use case? I think that would help us to determine exactly what it is that griddler is doing wrong (or sub-optimally) and how to make this work better.

calebhearth commented 11 years ago

Closing for inactivity. Feel free to reopen with failing test.