search

thunderbird / import-export-tools-ng

Import Export Tools that supports Thunderbird 68/78/91/102/115
Other
267 stars 30 forks source link

incorrect encoding when exporting to html #592

Open lili36 opened 1 week ago

lili36 commented 1 week ago

Hello ! it seems to work before, but on y last exports, the encoding in html is wrong... I always get broken char, if I export in iso8859-1 I get some wrong, if I export in UTF-8 I get other worng (also the initial html mail is supposed to be UTF-8 anyway... but quoted-printable transfer encoding

I get  before the start, then é and è for example in UTF-8 and "?" display with ISO-8859, because "Ú" is encoded as è (the è is not passing)

Also, I get 4 different UTF-8 lines in the option panel ? Is there no" autodetect" option ? The display of the mail is perfect in Thunderbird.

I can't find no way to have a correct encoded html export...

I use it on Thunderbird 115.12.0 (64 bits)

Thank you !

cleidigh commented 1 week ago

This should be fixed in v14.1.0 beta. You can download here : https://github.com/thunderbird/import-export-tools-ng/issues/575 Christopher

On Sun, Jun 30, 2024 at 7:41 PM lili36 @.***> wrote:

Hello ! it seems to work before, but on y last exports, the encoding in html is wrong... I always get broken char, if I export in iso8859-1 I get some wrong, if I export in UTF-8 I get other worng (also the initial html mail is supposed to be UTF-8 anyway... but quoted-printable transfer encoding

I get  before the start, then é and è for example in UTF-8 and "?" display with ISO-8859

Also, I get 4 different UTF-8 lines in the option panel ? Is there no" autodetect" option ? The display of the mail is perfect in Thunderbird.

I can't find no way to have a correct encoded html export...

I use it on Thunderbird 115.12.0 (64 bits)

Thank you !

— Reply to this email directly, view it on GitHub https://github.com/thunderbird/import-export-tools-ng/issues/592, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGAZ762KU45WHZVWX5S4QATZKCJS3AVCNFSM6AAAAABKEQFNOKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4DENJQGE3DSNA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

cleidigh commented 1 week ago

@lili36 Any chance you tested the beta? @cleidigh

lili36 commented 1 week ago

Hi ! Thank you for your answer ! not yet, because I was too busy (sick) and also because I don't know how to download and install it via github yet, so I need to check.

cleidigh commented 1 week ago

@cleidigh Hope you are feeling better! To download and install the xpi, do the following :

@cleidigh

lili36 commented 1 week ago

Thanks ! I will do that when I feel better (not yet)

cleidigh commented 1 week ago

@lili36 Chicken soup, the do all elixer %-) @cleidigh

lili36 commented 1 week ago

ahah ^^ is there any risk if I use a beat version (I am afraid of that, don't judge me I am not expert of all this, I just joined github to post the bug)

cleidigh commented 1 week ago

@lili36 Totally fair question. This beta is one cleanup away from release. I am 99% certain this is a duplicate of resolved

lili36 commented 1 week ago

I was thinking "any risk for my mails ?" (I already have a thunderbird bug telling me I could save 1,7To by compacting my mails, while my disk only has 200Go, so... (also .eml is not supposed to be compacted). I was afraid to add additionnal bugs or loose of mails).

I mean, I get a popup saing it wll give complete access to my Thunderbird and computer and I can't say "no to the pop-up" (I mean the install pop-up is bugged, I am restarting thunderbird). I guess you need it to export, but this means a bug in it can harm by computer files.

lili36 commented 1 week ago

I tried and just trying to install it makes my thunderbird crash... I get the popup "are you ok to give access to thunderbird/computer and I can't click on cancel or yes, both of them do nothing and I need to reboot thunderbird then.

lili36 commented 1 week ago

also, any option to auto-detect encoding to use to export ? I mean, I need to select UTF-8 to make it work in .txt, but ISO8859-15 to make it work in .hmtl (and there is everychoice in double in the liste)

cleidigh commented 1 week ago

@lili36 I'm a bit confused about your comments. The character set option on the misc tab is only for text and csv files as the label indicates. The html output is always utf-8 with bom. There is no conversion in the html save. If I do a binary comparison with html exports, changing the text export charset, the html outputs are identical.

How are you finding them different? Also what os and Thunderbird version are you using? Btw, I do see the duplicate entries, have to check that. @cleidigh

lili36 commented 1 week ago

Hi, Yes, changing this option (even if it says it is for text and cvs files) changes the way html is exported. If I select UTF-8, I get it all wrong because html is in ISO but saved in UTF-8, and half of the characters are wrong (even some sapces) If I select ISO-8859, I get less wrong characters in html file. It changes a lot, and I am certain this is this setting, I tested it multiple time. I can't show your a screen right now, I can try to take some next time.

The meta/head is quite long, maybe that is it (if I export in html with thunderbird, I get correct and also shorter and smaller file).

I am on Linux Mint 20.2 Uma using Thunderbird 115.12.0 (64 bits)

Also, I can't install the beta version (could you read my messages above the last one ?)

Thanks !

cleidigh commented 1 week ago

@lili36 ok let's back up. The v14.0.3 did have an error where charset encoding WAS being used for html, that would track your observations. V14.1.0 fixed html to utf-8 bom.

So apologies for not getting that you couldn't install v14.1.0 I have had some people that for whatever reason had issues downloading from github. Can you try a different browser? Also do an uninstall first. In a worst case scenario I will have to send you a dropbox link. Should not have to, hundreds don't have the issue with github. @cleidigh

cleidigh commented 1 week ago

Send me an email to my test account : test1@kokkini.net I will send you a dropbox link from my private email. I also retested b4 on my Linux mint, all results good.

@cleidigh

cleidigh commented 1 week ago

@lili36 I just realized I can post a zip. Let me know. import-export-tools-ng-14.1.0-b5-tb.zip

lili36 commented 1 week ago

Hello, I had no issue downloading the file ! I have the file, but the install in Thunderbird fails. I get the pop-up asking if I accept to give access to whoe thunderbird and computer, and then I am stucked. Clicking on "no cancel" or "yes" doesn't close it and I need to restart thunderbird. So this is not a downloading problem, I got it perfectly easily from github.

Or do I have to uninstall the v.14.0.3 first ? (this is still the version I have after many tries of installing the other, I don't know why it crashes. @cleidigh

cleidigh commented 1 week ago

@lili36 I am Confused as heck, never had Thunderbird crash/freeze on installing. So just for the record, what os are you on?

Let's try a couple things.

Then remove v14.0.3 Change to the main tab and close Thunderbird Open Thunderbird, make sure no ietng present. Close Thunderbird again and open again.

First let's open the debug console with Control Shift J Try installing beta again, if it fails, go to console If there is anything there, right click, copy all messages and then post here. Hopefully that will give us some clues. @cleidigh

cleidigh commented 6 days ago

@lili36 Any luck with my suggestions? @cleidigh

lili36 commented 2 days ago

Hello, I didn't want to try to not loose the parameters, thinking I may as well wait for the release. I tried to make a fresh profile to install yours and I am stucked too but... same thing with any other extensions... investigating...

[EDIT] : so this issue is general for Thundebird on Linux, for a year, Thunderbird didn't plan to fix it yet, but I found a workaround to test your v14 which worked for me (using "tab" key instead of mouse to validate the installation) So results with v14 are : body of the mail is better, but header is broken.

I pass from "De : Cécile " on v13 (current version) to "De : =?UTF-8?Q?C=c3=a9cile_" on v14

I tested on some other email : in source I have some Subject: =?ISO-8859-1?Q? But Subject is well displayed However To: =?ISO-8859-1?Q? leads to display of ?ISO-8859-1?Q?+wrong encoded names in html file.

Content is fine.

Also, doing .eml export, I see that now subject are truncated even if I deselect the "cut at 50 char" option, is that normal ? It is fine with txt and html. But .eml truncated the name of file at 68 characters, whatever my settings are. Also weird things : This morning, I had longer names. This afternoon, after testing new v14 beta, I have this shorter 68 char names, and can't come back even in previous version.

previous v14 (on release) was already truncated compared to old exports I made in march 2022.

@cleidigh

lili36 commented 2 days ago

@cleidigh Since I edited my message many times, not sure if you see the updated or how your alert works exactly, so new message (sorry)

cleidigh commented 2 days ago

@lili36 I did get all your posts. That is good you found the workaround. It did seem to be a Thunderbird bug. Can you send me sample emails with the issues? Use test1@kokkini.net I was going to release today, but I will wait a day or so if this has a speedy solution. Let me know your locale also. I will also check the truncation. @cleidigh

lili36 commented 2 days ago

@cleidigh mail sent. The truncation is off again in new version, and on on old, this is weird. Not sure if this is linked to the fact that I now use 2 thunderbird profils with 2 versions...I have trouble to reproduce since it vary in time/according to mails.

As I said in my mail, export is failing when there is Mozilla status on top of source code (from mbox to eml) and seems fine if I export directly from IMAP's folder eml.

cleidigh commented 2 days ago

I think I have the truncation issue found, part of it is the dueling pref for cut to 50. I am removing that from the ui. What do you mean about the Mozilla-Status? it was not clear on the operation? @cleidigh

lili36 commented 2 days ago

@cleidigh details are in my mail.

Step I did : 1) send a mail to myself 2) receive it, and export it directly from IMAP folder -> 1.eml and if I export in html it works fine 3) move this mail to local folder -> it adds some mozilla status and gives 2.eml as a result. Then the html export from this local folder fails.

1.eml and 2.eml are attached to the mail, but you probably can just reproduce like that.

cleidigh commented 2 days ago

I am testing truncation, have not gotten to email. What time will you be online until?

lili36 commented 2 days ago

I am going to bed right now... but I hope to be still alive tomorrow :D

cleidigh commented 2 days ago

ok I will see what I can address by tomorrow

cleidigh commented 1 day ago

I fixed the truncating issue. I can't reproduce the local folder issue, used same method and your email... spent hours on encoding, keep getting half results