search

mjl- / mox

modern full-featured open source secure mail server for low-maintenance self-hosted email
https://www.xmox.nl
MIT License
3.7k stars 111 forks source link

Import issues - encoding - broken diacritics #249

Open exander77 opened 1 week ago

exander77 commented 1 week ago

I have imported 90k e-mails to test and a lot of them has broken encoding in WebUI:

d�kuji v�em za rychl� hlasov�n�. Vypadl z toho term�n

Most likely related to this:

Content-Type: text/plain; charset=iso-8859-2

WebUI has to convert all data to UTF-8 to show.

mjl- commented 1 week ago

Is this a text/plain or a text/html part that is being displayed incorrectly? Text parts are parsed server-side (from whatever the content-type message header says into utf-8), and a JSON is sent to the webmail, which renders the text. HTML parts are parsed (also decoding from content-type in message header into utf-8), sanitized, and sent to the browser for rendering. Something is probably behaving badly. Would you have a test message that shows this behaviour? You could also send it privately to me, at mechiel@ueber.net.

exander77 commented 1 week ago

Both text and HTML view is broken exactly the same way.

mjl- commented 1 week ago

And any chance you can provide an example of a message that is not being handled correctly? Could be a modified/minimal message that shows the behaviour.

Raw messages are easily exported from the webmail interface. After modifications (e.g. removing sensitive information), it can easily be imported again for testing with "mox deliver you@domain < message.eml".

exander77 commented 1 week ago

Sent you an e-mail.

mjl- commented 1 week ago

Thanks, sent a reply with some next steps to investigate.