v14.0.1 Release Status - Beta Download

[cleidigh]

v14.0.1 has been posted to the review Q! Thanks to everyone!

If you get a corrupted message on install make sure you follow the download method here: https://github.com/cleidigh/printing-tools-ng/issues/154

https://github.com/thundernest/import-export-tools-ng/blob/v14.0.1/xpi/beta/import-export-tools-ng-14.0.1-b8-tb.xpi

Status:

b1

• New - import and export mboxes up to 30GB!
• fix csv output to text not HTML

• 451 , #463

• fix export folder to zip
• fix help to not load on Thunderbird updates
• add preference to never load help on any updates

• 458

• extensions.importexporttoolsng.export.help.showOnInstallAndUpdate
• add back messages menu to display context

• 459

• add warning for multiple messages selection for copyToClipboard
• complete de locale update - @Mr-Update
• fix Headers translation de locale - @acbugreport #447
• complete fr locale update - @DenB10
• ja locale update - @kiki-ja

b2

• fix account export for pop3
• fix illegal filename character substitution for export
• fix zip export with ascii only option
• fix ja messages, was fr
• de locale update , typos in js - @Mr-Update
• fr locale update - @DenB10
• dark mode css - @DenB10
• null out Thunderbird From_ separators

b3

• Important mbox import fix for boundary From_ escaping
• make mbox zip files unique
• use msgctx menu only in msgDisplay
• fix copyToClipboard when in msgDisplay msg menu
• use -1 suffix on es6 module to force reload
• complete da locale update - @Joergen

b4

• New - export mbox flattened at Account Level
• New - import mboxes at Account Level (Local Folders)
• fix msgDisplay ctx menu export
• fix page ctx hiding msgCtx menu #489
• fix right click on unselected message #485
• fix help for sublocales and unsupported locales
• only load help on extension updates, not Thunderbird
• ja, de, fr, da locale updates

b5

• redo selected messages handling

b6

• add release notes to help
• final cleanup
• release candidate

b7 - will release unless major issue report

• add open help in tab option
• fix help pref
• make options non-modal
• help tweaks
• final final cleanup

b8

• filter cmd dispatch on focused window

[Yarny0]

Thanks for the quick implementation, @cleidigh.

I tested most export functions and formats, and everything worked as I assume it should work, with these exceptions:

• Exporting a complete mail account just generates an empty directory. I tested this with the context menu of the account's entry in the folder pane, where it says "Export Accont - (ACCOUNT-NAME)". The Error Console logs the following message:

  Uncaught (in promise) Error: Component returned failure code: 0x80550006 [nsIMsgFolder.messages]    mboxImportExport.js:507
    buildAndExportMbox chrome://mboximport/content/mboximport/modules/mboxImportExport.js:507
    exportFoldersToMbox chrome://mboximport/content/mboximport/modules/mboxImportExport.js:455
    exportAccount chrome://mboximport/content/mboximport/mboximport.js:1002

• If both options "Use .mbox extension for mbox files" and "Use just alphanumeric characters in export" are enabled, exporting a folder as "Single Zipped mbox File" generates an incomplete zip file (it's just 22 Bytes in size, it's probably just a zip header). The Error Console logs the following message:

  10:29:08.910 Uncaught (in promise) NS_ERROR_FILE_NOT_FOUND: Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIFile.isDirectory]     mboximport.js:855
    IETexportZip chrome://mboximport/content/mboximport/mboximport.js:855

  It also generates a file (FOLDER-NAME)_mbox in the same directory, which contains the unzipped mbox. The export process maybe gets interrupted before it can zip and then remove the unzipped file.

For these tests I used Thunderbird 115.3.1 (on NixOS 23.05) with a fresh profile, with all settings at their default and only ITE (14.0.1-b1) installed as addon. I created a POP3-connected mail account with two small messages in its Inbox to test exports.

(Note: I'm not relying on mbox or full account export functions. My users mainly need IET for EML-export of individual messages. But I'd like to help development with testing the beta version. I hope I'm helping with this report. Thanks for IET!)

[DenB10]

Hi, I can't confirm the first comment, but I can confirm the second. Everything else works fine for me except the French translation, which has not been fully updated (in the _locales folder and the help file). In addition, the clipboard copy warning window should be translated when several messages are selected! Best regards, @DenB10

[cleidigh]

@DenB10 Hope your vacation was good! I think I must have merged the wrong zip from you, I will verify and remerge. On the clipboard warning, that actually is not translated yet (still static English) so neither of us missed that. Will do when I have the next batch. Thanks again for the effort, vive la France %-) @cleidigh

[cleidigh]

@Yarny0 Glad the help options get you back into your workflow. Your report and interest in helping the development beyond even what you utilize is so helpful!!

@DenB10 @Yarny0 ImportExportTools is a really difficult tool because it not only has to deal with the ever changing Thunderbird (v115 required really major changes). It also has to deal with many external data that most of the time I cannot see. I am but one, disabled person so I do my best, however, its help from people like yourselves that makes this possible. Thanks again! Christopher

[cleidigh]

@Yarny0 @DenB10 On the account export : I had one earlier report. This is a fundamental failure where I am asking for the message iterator for each folder to export one by one. The process completely fails because Thunderbird throws an error. This is something that I am worried will be tricky to figure out.

@DenB10 @Yarny0 Were both of your accounts pop type where you saw this failure?

The second issue should be a basic fix. @cleidigh

[DenB10]

@cleidigh

Hope your vacation was good!

My vacation went well, thank you! Except that it was really too crowded where I was (Florence in Italy). Mostly insufferable people whose sole aim was to take a photo of themselves in front of "Primavera" from Botticelli with a thumbs-up. I would have liked to answer them with my middle finger raised if I had their education...

vive la France %-)

I'm not patriotic because the idea of homeland is a pretext for war. I'd rather be a citizen of the world, and when everyone is (in vain hope), we'll see who the real enemies are.

But back to IET NG:

I think I must have merged the wrong zip from you, I will verify and remerge.

Don't worry, I'll propose changes directly in the v14.0.1 branch by pull request if you prefer. Especially as the translation will evolve with the future changes.

Were both of your accounts pop type where you saw this failure?

My accounts are all imap accounts and I don't have the first problem Yarny0 mentioned with exporting an account. I simply have the same remark (second point) that he indicates for exporting a folder.

Last point I would like to make: I think that "msgDisplayCtxMenuSet" is no longer needed in the "wextMenus.js" file because it's redundant. What do you think?

[cleidigh]

@DenB10 Florence is wonderful, minus the tourists. When I was able to travel my favorite thing to do was find the places and paths less traveled. You are right about nationalist phrases, I am no "go USA" guy, and today especially with conflict everywhere...

but back to ietng as you said.

Doing a PR for the translations would be excellent as it makes things easier and less error prone. Two things. First do the PR against the branch not master. Second give me a heads up in case I am about to add or change translations.

I verified the zip issue. Another byproduct of the mbox extension. Added that for another user last minute, bad idea. @cleidigh

[cleidigh]

@DenB10 Not sure what you mean about the menus, they need unique ids. If you mean the fact that the copyToClipboard functionality is in the msgCtx menu that's a bit of a competing interst, v115 versus old menu conflict thing. Not sure how to satisfy everyone. @cleidigh

[DenB10]

I meant that the "Copy to clipboard" entries were duplicated in the page context. But after several attempts, I find that the "Copy to clipboard" commands built into "Export messages as..." don't work in the page context.

[cleidigh]

@DenB10 Exactly, some users wanted the message menu in the display and this not only creates a conflict, but moves the menus one more level down. I have to figure out a better way, but someone won't be happy. @cleidigh

[DenB10]

I have to figure out a better way, but someone won't be happy.

Yes, you have to make a choice, but I'm sure you'll make the right one ;-) Note: I understand that an extension can't have two root entries in the context menu, is this true?

[DenB10]

@cleidigh

I've made two PRs for the French translation: one for the "importexport-help.html" file and one for the "messages.json" file.

[cleidigh]

@DenB10 ha ha, it will be my way or the highway! Yes you are correct, one menu entry per extension. Also extensions at the bottom only %-{ @cleidigh

[Yarny0]

Were both of your accounts pop type where you saw this failure?

Yes, mine was a POP3.

To be sure, I tried again, again with a fresh Thunderbird profile, complete default settings, IET beta version installed, also with default settings. I created two mail-accounts: One with IMAP-connection, one with POP3-connection. I sent two emails, so each account had one mail in its "Sent" folder and one mail in its "Inbox".

Exporting the IMAP account causes a warning by IET that not all messages might be present, but then worked flawlessly: It created an mbox-file for each folder. Exporting the POP3 account does not show a warning, but just yields an empty folder.

[cleidigh]

@Yarny0 @DenB10 Found the TB issues with the pop account export. Should be good now. Also fixed the ascii option issue. @cleidigh

[Yarny0]

Hi @cleidigh, I did again a thorough test of all export functions.

• This is a rare case, but it might lead to surprising data loss:

  When I use the export as "Single Zipped mbox File" of a folder, IET first creates an mbox file in the target folder and uses the folder name as filename. If a file with that name already exists, a number is appended to the filename (so I observed) to make it unique. After the mbox file is set up, it gets zipped. The zip file is named after the mbox file by appending .zip.

  However, this step overwrites an existing zip file with the same name, without warning. So exporting many Inbox folders "As mbox File" leads to many files Inbox.mbox, Inbox-1.mbox, Inbox-2.mbox, while exporting many Inbox folders as "Single Zipped mbox File" will yield just one Inbox.mbox.zip containing the messages of the last folder that is exported.

• Note: I somehow managed to see the full folder export menu in the context menu of an account. It was consistently showing up for an IMAP and a POP3 account, even when I exported single messages or folders in between. In that case, a "Single Zipped mbox File" (applied to a complete account) creates an empty zip file, and an export "As mbox File" creates the same directory structure that "Structured With Subfolders" does.

  I was already about to report that observations as bug, but when the menu suddenly switched to "Export Account - ...", I realized that the full export menu is probably not supposed to be available on an account.

  I am currently failing to reproduce this phenomen, even with another fresh profile. Nevertheless, I'd like to document it here; maybe this information turns out to be useful if someone else sees a similar problem, or you even happen to find a problematic code line that might cause such behaviour.

  If I find a way to reliably reproduce this behaviour, I will report back.

Other than that, all export functions do what I would expect them to do. For the record: I tested with

• IET 14.0.1-b2 with Thunderbird 115.3.1 (on NixOS 23.05)
• two fresh thunderbird profiles, one with mbox-accounts, one with maildir-accounts
• in each profile: one IMAP-account with two Inbox-messages (one unread, one with attachment), one POP3-account with two Inbox-messages (one unread, one with attachment)
• IET options at their default values, with these exceptions
  • "use .mbox extension for mbox files" enabled
  • "HTML format as displayed in messages panel" enabled
  • "use just alphanumeric characters in export" enabled

Again, thank you very much!

[cleidigh]

@Yarny0 Your testing and reporting is much appreciated especially as your main concern is eml export!

I have it on the b3 todo list to make the zip file unique. Adding the mbox extension option at the request of a user at the last minute was a mistake as had long tentacles.

On the menus, this is something I have seen myself maybe twice. I am a little dubious of the menu api, however, I do have to see if there are any holes in my menu update functions.

And good year pop3 exports are working. @cleidigh

[mforkel]

While it seems to work for other people, I couldn't manage to install import-export-tools-ng-14.0.1-b2-tb.xpi with the "Install Add-on from file..." command on the Add-ons management page of Thunderbird 115.3.3 (64 bit). I get a warning that the add-on couldn't be installed because it seems to be damaged. (Please note: This is my first attempt at installing a Thunderbird extension from a file, plus I'm on a German Thunderbird, so command name might differ.) Any suggestions?

[cleidigh]

@mforkel Don't worry, nothing to do with German %-) It's a weird Github issue. Here is a note on this : https://github.com/cleidigh/printing-tools-ng/issues/154

@cleidigh

[librelegal]

b3: Windows: Tested profile export, worked fine (~6,5 GB), but I count 3 files less than original. Tested export of mbox file, but the file name was not .mbox, ended without dot and extension. Tested to single zip file, in which the file name was not .mbox, ended without dot and extension. Linux: Tested profile export, worked fine. Tested export of mbox file, but the file name was not .mbox, ended without dot and extension.

[cleidigh]

@librelegal Many thanks!

• Did you id the three files
• When you did the exports, you had the mbox extension option checked?

I did a ton of testing on the naming, also did you restart? I may still have a cache issue @cleidigh

[librelegal]

Sorry, no I could not identify the three files that according to Windows were missing in the profile export in comparison to the original profile folder.

Yesterday did not first check the mbox export option. I had simply chosen the inbox folder within TB, did a right click, chose to export the folder in the mbox format and then decided on the location. The file was called "inbox" and not "inbox.mbox".

Today with 115.4.1 and obviously after a restart on Windows I repeated the mbox export after I had first checked the mbox option and chosen the folder. The result was the same: the file was called "inbox" and not "inbox.mbox".

[cleidigh]

@librelegal Very odd, I cannot reproduce. Could you try exporting a folder to zip and capture the console? This has some debug I am curious to see. Unfortunately this mbox extension, simple as it should be is a pain... Thanks @cleidigh

[Joergen]

Does it help to select "Use .mbox extension for mbox files (non-structured)" in the Misc tab?

[cleidigh]

@Joergen Were you able to do an export with the mbox extension option and get an x.mbox result? @cleidigh

[Joergen]

@cleidigh Yes - on a Linux computer. Without a check mark next to "Use .mbox extension for mbox files (non-structured)" the add-on does not append the file extension to the filename.

[DenB10]

Hi everybody, I agree with @Joergen I'm on Windows and I don't have any problems with mbox export either.

[cleidigh]

@Joergen Yes that is what the new option is for. The default is no extension because that is the Thunderbird standard (no mbox extension) therefore until I added the option IETNG has always followed Thunderbird.

So I have windows working with the extension, you have Linux working, but @librelegal is not getting the extension. Perhaps we have another preferences conflict. @cleidigh

[cleidigh]

@DenB10 Excellent, so two platforms (three locales) provides confidence we have a configuration or update issue. @cleidigh

[librelegal]

It DOES work on Windows, I do get a file called x.mbox. Truely sorry, I thought I only had to tick this

... but I also had to tick this: (maybe it would be better to automatically tick this, when chosing to export a folder in that format?)

and

[librelegal]

And it does work on debian 11, KDE. Yippee Yeah

[librelegal]

@librelegal Very odd, I cannot reproduce. Could you try exporting a folder to zip and capture the console? This has some debug I am curious to see. Unfortunately this mbox extension, simple as it should be is a pain... Thanks @cleidigh

And under Windows a .zip works, it does contain an .mbox

[cleidigh]

@librelegal Good, ok I was worried. As I described to @Joergen above Thunderbird and therefore IETNG have always had NO extension so I am hesitant to change the default, however, when I make both structured imports and exports work with the mbox extension I will poll everyone to see if we have agreement.

@librelegal @DenB10 @Joergen Also a related naming issue : The option Use only alphanumeric characters in filenames only allows a-z 0-9 and dash, all other characters replaced with underscores. I discovered even the period. This may have made sense before, however, I have to change that. @cleidigh

[cleidigh]

@librelegal Our posts crossed... So to be clear, both Windows and Linux export with the extension as long as the option is checked. So all is good? @cleidigh

[librelegal]

@librelegal Our posts crossed... So to be clear, both Windows and Linux export with the extension as long as the option is checked. So all is good? @cleidigh

Yep. Can I suggest that when exporting a folder as mbox it should be exported with that extension even if the default for files would not be .mbox?

[cleidigh]

@librelegal Super bueno! Have to make sure one of best customers is cool %-} @cleidigh

[librelegal]

Smile. Here is a suggestion for the file name of these .mbox files, #486

[Yarny0]

For the record: I tested beta3 with a fresh profile, with mbox mail storage, with one IMAP and one POP3 account, with emails with and without attachment.

• All export formats work flawlessly (I didn't test each function, but at least one export function for each format, and at least one attachment exporting function for each format that supports such). In particular, I no longer can provoke IET to acccientally overwrite an existing file.
• Import of multiple eml files and of a single mbox file work also flawlessly.
• Full export of an account also works as expected. I no longer observed the wrong menu in the account's context menu -- I'm not sure if this is luck or is fixed (cf. https://github.com/thundernest/import-export-tools-ng/issues/466#issuecomment-1771305525).

This is a great version!

[DenB10]

I agree, all that remains to be done is to localize certain texts that are not yet translatable.

[cleidigh]

@DenB10 My head is all over the place... Do you know what translations are missing? I thought I had done them all. @cleidigh

[DenB10]

@cleidigh Unless I'm mistaken :

• Lines 1974 and 1975 of the exportTools.js file ("Copy Headers to Clipboard" and "Multiple messages selected, only fist message headers copied to clipboard") as well as lines 1903 and 1904 are not translated.
• In ImportMboxModule.js: lines 87 to 94 (not tested as I don't have a mbox larger than 4 GB).

[Mr-Update]

@cleidigh Missing translation also exists in mboxImportExport.js und mboxImportExport-1.js (line 119):

• "This mbox exceeds the 4GB direct import size.\n\nDo you want to use the copy import method ?\n\nThis method will not do mbox processing.\nIf the mbox has not been processed, some messages may\nbe corrupted."

[cleidigh]

Translators: @kiki-ja @DenB10 @Mr-Update @Joergen If possible it would be much appreciated for these entity translations :

"Copy Headers to Clipboard" "Multiple messages selected, only fist message headers copied to clipboard." "Copy Messages to Clipboard" "Multiple messages selected, only the fist message copied to clipboard."

And I would like to thank you all again for making IETNG truly well localized! @cleidigh

[Mr-Update]

@cleidigh "Kopfzeilen in die Zwischenablage kopieren" "Mehrere Nachrichten ausgewählt, nur die ersten Nachrichtenkopfzeilen in die Zwischenablage kopiert." "Nachrichten in die Zwischenablage kopieren" "Mehrere Nachrichten ausgewählt, nur die erste Nachricht in die Zwischenablage kopiert."

[DenB10]

@cleidigh

"Copy Headers to Clipboard" > "Copie des en-têtes du message dans le presse-papiers" "Multiple messages selected, only fist message headers copied to clipboard." > "Plusieurs messages sont sélectionnés, seuls les en-têtes du premier seront copiés." "Copy Messages to Clipboard" > "Copie du message dans le presse-papiers" "Multiple messages selected, only the fist message copied to clipboard." > "Plusieurs messages sont sélectionnés, seul le premier sera copié."

And I would like to thank you all again for making IETNG truly well localized!

You're welcome!

[kiki-ja]

@cleidigh

If possible it would be much appreciated for these entity translations :

See Issue below for Japanese translation and update.

ja locale update · Issue #461 · thunderbird/import-export-tools-ng · GitHub https://github.com/thunderbird/import-export-tools-ng/issues/461

@kiki-ja

[Joergen]

@cleidigh

Here are the Danish translations: "Copy Headers to Clipboard" -> "Kopier headers til udklipsholder" "Multiple messages selected, only fist message headers copied to clipboard." -> "Flere meddelelser valgt, kun første meddelelses-header kopieret til udklipsholder." "Copy Messages to Clipboard" -> "Kopier meddelelser til udklipsholder" "Multiple messages selected, only the fist message copied to clipboard." -> "Flere meddelelser valgt, kun første meddelelse kopieret til udklipsholder."

I found a machine translated string that I missed the first time. If you could change the following string from the Danish mboximport.dtd file: <!ENTITY useMboxExt.label "Brug .mbox-udvidelsen til mbox-filer (ikke-struktureret)"> to <!ENTITY useMboxExt.label "Brug filendelsen .mbox til mbox-filer (ikke-struktureret)">

[cleidigh]

@kiki-ja @DenB10 @Joergen @Mr-Update Thanks all. Should have b4 out tomorrow and that should be good except if we want translation for release notes in help, thoughts? @cleidigh

[Yarny0]

Hi, I tested all available export functions again. Most of them work flawlessly, but I discovered the following issue:

When exporting messages (context menu after selecting two folded messages in the message list) as "New mbox file" from a POP3-account with "maildir" storage, no file is created, and the error console reports:

TypeError: uri is undefined   MailServices.jsm:20:17

I'm not sure if mbox export is supposed to be presented to the user here at all. As I suspect this problem to be connected with the "wrong context menu in folder tree" bug (where only mbox export is offered for maildir folders, cf. https://github.com/thunderbird/import-export-tools-ng/issues/466#issuecomment-1771305525 ),

I made precise notes on how I set up Thunderbird before I reached the broken export function

Most of the following is probably over-specific (like servernames or passwords), but I want to maximize reproducibility of the bug. * Start Thunderbird with a fresh profile * Go to "Add-Ons and Themes" * Disable "Update Add-Ons Automatically" in the gearwheel menu * Install `import-export-tools-ng-14.0.1-b4-tb.xpi` via "Install Add-On From File..." the gearwheel menu; acknowledge permissions with "Add"; acknowledge installation with "OK" * Go to Settings, "General" section * Set "Message Store Type for new accounts" to "... (maildir)" * Go back to the "Account Setup" tab * Create account: * name `user1`, Email `user1@localhost`, Password `1`, "Remember" checked * use "Configure manually" * Incoming Server: "POP3", `localhost`, Port `110`, Security "None", Authentication "Normal pssword", User `user1` * Outgoing Server: `localhost`, Port `25`, Security "None", "No authentication" * "Done" * Confirm the risks of unencrypted connections * Note: user1 has two mails (each with one attachment) in its mailbox. One message is a follow-up of the other, so Thunderbird shows these messages "folded". * close and restart thunderbird (Note: I'm doing this to create a copy of the profile folder to repeat tests without having to do the setup again) * click on user1's Inbox * right-click on the only folded message * select "Export Messages As...", "mbox Format", "New mbox File" * Enter path `/tmp/abc`, "Save"

Note that I also observed the "wrong context menu in folder tree" bug again, but alas, I still cannot reproduce it sigh .

Thanks again for this add-on.

[DenB10]

Hi, there,

I don't have the same problem as @Yarny0 (my accounts are imap accounts) but I do have another problem: The context menu of an account offers with version b4 two new items: "Copy Folder Path" and "Open Folder Directory" which do not work.

N.B.: thank you @cleidigh for allowing the help to be opened in a window, which makes it much easier (for me) to access the help.