v14.0.1 Release Status - Beta Download

cleidigh commented 1 year ago

v14.0.1 has been posted to the review Q! Thanks to everyone!

If you get a corrupted message on install make sure you follow the download method here: https://github.com/cleidigh/printing-tools-ng/issues/154

https://github.com/thundernest/import-export-tools-ng/blob/v14.0.1/xpi/beta/import-export-tools-ng-14.0.1-b8-tb.xpi

Status:

b1

New - import and export mboxes up to 30GB!
fix csv output to text not HTML
451 , #463
fix export folder to zip
fix help to not load on Thunderbird updates
add preference to never load help on any updates
458
extensions.importexporttoolsng.export.help.showOnInstallAndUpdate
add back messages menu to display context
459
add warning for multiple messages selection for copyToClipboard
complete de locale update - @Mr-Update
fix Headers translation de locale - @acbugreport #447
complete fr locale update - @DenB10
ja locale update - @kiki-ja

b2

fix account export for pop3
fix illegal filename character substitution for export
fix zip export with ascii only option
fix ja messages, was fr
de locale update , typos in js - @Mr-Update
fr locale update - @DenB10
dark mode css - @DenB10
null out Thunderbird From_ separators

b3

Important mbox import fix for boundary From_ escaping
make mbox zip files unique
use msgctx menu only in msgDisplay
fix copyToClipboard when in msgDisplay msg menu
use -1 suffix on es6 module to force reload
complete da locale update - @Joergen

b4

New - export mbox flattened at Account Level
New - import mboxes at Account Level (Local Folders)
fix msgDisplay ctx menu export
fix page ctx hiding msgCtx menu #489
fix right click on unselected message #485
fix help for sublocales and unsupported locales
only load help on extension updates, not Thunderbird
ja, de, fr, da locale updates

b5

redo selected messages handling

b6

add release notes to help
final cleanup
release candidate

b7 - will release unless major issue report

add open help in tab option
fix help pref
make options non-modal
help tweaks
final final cleanup

b8

filter cmd dispatch on focused window

cleidigh commented 1 year ago

@Yarny0 @DenB10 Once again kudos for all the testing help!

@Yarny0 I have to mimic your setup (easy with your notes) for the mbox export. It's probably just a menu parameter thing, not too concerned. The big one is showing the full folder context on the account. I have seen this myself so we are not crazy. I have tried all sorts of things and cannot reproduce either. I have a large and dynamic menu and I am weary of the api, but not sure how to debug yet.

@DenB10 Yup I caught that AFTER posting b4. May remove them. As far the help in a window , you must have been digging. You can thank @kiki-ja , it was his idea and next rev will get into options. @cleidigh

cleidigh commented 1 year ago

@Yarny0 I just found what might be a related issue. My main test setup uses my imap test account and using my other addon LocalFolders I have multiple LFs one is setup as maildir. I just found that if I right click on the maildir LF the imap account and its folders now show the full mailder menu. So my question is do you always use maildir and or do you think the ctx errors were always with maildir. And, did the full ctx menu on the account show the full maildir or mbox menus.

I think I have something to work with because this state becomes sticky which is different than your observations and mine prior. @cleidigh

Yarny0 commented 1 year ago

do you always use maildir

In my productive Thunderbird setups I always use maildir. When testing IET, I try to mix storage types for more thorough testing.

and or do you think the ctx errors were always with maildir.

No, it seemed to me that it depends on what menues are opened in which order.

I just tested again, this time only with one message and without attachment (also POP3/maildir account). Context menu of the mail (not folder) offered mbox export (and eml/html/pdf/..), and mbox export of the message works. Then I created an answer to that message so that thunderbird sees two messages which are displayed "folded". Now, the mbox export fails. Unfolding the messages and mbox-exporting each with its context menu works. Folding them again, then using the mbox-export with the context menu of the folded message fails.

So I guess the issue is triggered by using the context menu on "folded" messages.

And, did the full ctx menu on the account show the full maildir or mbox menus.

I am 85% sure that it only showed the mbox menus, i.e. a menu item with four submenu items. I think I have never seen the maildir menu where I expected the mbox menu, but I have seen the mbox menu where I expected the maildir menu.

I might be wrong.

cleidigh commented 1 year ago

@Yarny0 I can reproduce the folded messages issue, however, with one specific aspect:

First select a single non-folded message then right click on a folded set and try any message export and you get only the first of the folded messages. If I directly only select the folded message I get the thread for new mbox or other types. Does that match your observations? I have to redo the selection handling. @cleidigh

Yarny0 commented 1 year ago

First select a single non-folded message then right click on a folded set and try any message export and you get only the first of the folded messages.

Not exactly, it seems to depend on which message is selected (blue background). I tried this (POP3/maildir) with three messages in the inbox, two of them folded, the third one being unrelated to the other messages:

left-click on the single message
right-click on the folded messages, select eml or mbox export
result: the single unfolded message gets exported

Then I tried this:

left-click on the single message
left-click with ctrl on it again, so that no message is selected
right-click on the folded messages, select eml or mbox export
result: nothing gets exported, the error console shows the type error

I conclude that IET always works on selected messages. If no message is selected, it fails due to the lack of selected messages. In my experiment described in https://github.com/thunderbird/import-export-tools-ng/issues/466#issuecomment-1792896562 , I apparently did not select any messages after starting thunderbird and went directly to the message list context menu to export something; in that case, nothing is selected.

Thinking loud (of couse you don't have to follow my thoughts): From a user experience perspective, I assume it is not unreasonable to work on selected messages only. I'm not sure what I (or any user) would expect from selecting message A and then right-clicking on message B to initiate an export. In any case, the menu item should indicate what is about to happen ("export selected messages" or similar).

If I directly only select the folded message I get the thread for new mbox or other types.

I'm not sure if I understand that question. In the situation described above (three messages, two folded, POP3/maildir), the context menu in the message list always looks the same, irrespective of which messages are un/folded or de/selected: It shows the options EML (submenu), HTML (submenu), PDF, plain text (submenu), CSV, mbox (submenu).

cleidigh commented 1 year ago

@Yarny0 I was able to reproduce your unselected to folded no output example. Basically our variety of selection and folded quirks are all related to the fact that I have two worlds I am dealing with, the new web extension apis and "legacy" world.

Each has different methods and forms of what they report on selected messages. I use a mix of the two which has some of these discrepancies based on not recognizing how ech method reports. So I have already plotted an improvement to use the exact reporting of the wext side for all but selections under 100 messages. The scenarios where more than 100 messages are selected can then be handled by the legacy selection views. Ultimately I will have to rely purely on the wext side or I will run into approval issues.

Should have my tweak by the end of day. And will post as b5. This should close the last known issue so I hope we can spin out v14.0.1. @cleidigh

cleidigh commented 1 year ago

@Yarny0 All

b5 I completely redesigned the selected messages handling now that I understand better what was going on. Essentially I guarantee to use the wext side to handle the selected messages except for selections over 100. All the quirks get handled this way.

@cleidigh

cleidigh commented 1 year ago

Translators: @kiki-ja @DenB10 @Mr-Update @Joergen

We are there!! B5 has the last patch. I am going to add the release notes to the top of help. I can auto translate so if people don't have time, no big deal.


Release Notes: v14.0.1

- New: Add Account level mbox import
- New: Add Account level mbox flattened export
- New: Support for mbox import and export to filesys levels
- Fix CSV export to text #451, #463
- Fix export folder to ZIP
- Fix help load only on IETNG updates
- Add preference to never load help on updates #458
  extensions.importexporttoolsng.export.help.showOnInstallAndUpdate
- Fix help for sublocales and unsupported locales
- Fix mbox From_ separators to adhere to RFC 4155 #455
- Add back Export Messages in message display windows #459
- Fix Account export for pop3
- Fix buffer boundary From_ escaping
- Fix selected messages handling #485
- Complete de locale update @Mr-Update
- Complete fr locale update @DenB10
- Complete da locale update @Joergen
- Updated ja locale @kiki-ja

Thanks as always @cleidigh

DenB10 commented 1 year ago

@cleidigh

I am going to add the release notes to the top of help.

Not knowing exactly where to insert the release notes in the help file, I'm providing the raw French translation below:

Notes de version 14.0.1
- Nouveau : ajout de l’importation mbox au niveau du compte
- Nouveau : ajout de l’exportation mbox à plat au niveau du compte
- Nouveau : prise en charge de l’importation et de l’exportation mbox au niveau du système de fichiers
- Correction de l’exportation CSV en texte (#451, #463)
- Correction de l’exportation des dossiers en ZIP
- Correction : chargement de l’aide uniquement lors d’une mise à jour de IETNG
- Ajout de la préférence « extensions.importexporttoolsng.export.help.showOnInstallAndUpdate »
  permettant de ne jamais charger l’aide lors d’une mise à jour (#458)
- Correction : aide pour les sous-langues et les langues non supportées
- Correction : séparateurs mbox « From_ » afin de respecter RFC 4155 (#455)
- Réintégration du menu « Exporter les messages » dans le panneau d’affichage des messages (#459)
- Correction de l’exportation d’un compte pop3
- Correction : échappement de « From_ » en limite de la zone tampon
- Correction de la gestion des messages sélectionnés (#485)
- Mise à jour complète de la locale « de » (@Mr-Update)
- Mise à jour complète de la locale « fr» (@DenB10)
- Mise à jour complète de la locale « da » (@Joergen)
- Mise à jour de la locale « ja » (@kiki-ja)

I also have a practical request before the final version is released:

Is it possible that the options window is no longer "modal"? When you call up help from the options page (via the question mark icons), the help is opened in a tab and cannot be accessed because it is blocked by the options window, which remains in the foreground. (I hope you've understood what I mean?)

cleidigh commented 1 year ago

@librelegal In b5 I reimplemented the handling of selected messages to deal with issues involved with folded threads and a couple other items. This effects the exportSelectedMessages() interface you use with shortcuts. I tested with IETNG shortcuts. If you have a moment to check I want to make sure you are still good. Also note you have been added to the infamous IETNG crowd in help(rel version pending)

466

cheers @cleidigh

cleidigh commented 1 year ago

@DenB10 Merci... All integrated, you can see in b6. Almost ready for release. https://github.com/thunderbird/import-export-tools-ng/blob/v14.0.1/xpi/beta/import-export-tools-ng-14.0.1-b6-tb.xpi @cleidigh

I missed your last comment. I think that unfortunately may be alot hrder than it sounds. I think getting the option to open help in a window will be something we could do next cycle since we already have that pref

librelegal commented 1 year ago

@librelegal In b5 I reimplemented the handling of selected messages to deal with issues involved with folded threads and a couple other items. This effects the exportSelectedMessages() interface you use with shortcuts. I tested with IETNG shortcuts. If you have a moment to check I want to make sure you are still good. Also note you have been added to the infamous IETNG crowd in help(rel version pending)
* [v14.0.1 Release Status - Beta Download #466](https://github.com/thunderbird/import-export-tools-ng/issues/466)

Dear Christopher, the exportSelectedMessages() thing works with B6. I thank you a lot for also respecting that need.

kiki-ja commented 1 year ago

@DenB10

When you call up help from the options page (via the question mark icons), the help is opened in a tab and cannot be accessed because it is blocked by the options window, which remains in the foreground.

ImportExportTools NG 14.0.1-b５, b6 Thunderbird 115.4.2 Windows 10 Pro 22H2

Workaround Method: How to Open Help in a Window

Select an account, folder, message, or the message pane.
Right mouse click.
Click Help on the context menu.

Caution: This method may be changed in future versions.

@kiki-ja

DenB10 commented 1 year ago

@kiki-ja

Workaround Method: How to Open Help in a Window

Select an account, folder, message, or the message pane.

Right mouse click.

Click Help on the context menu.

Yes, I know that, but, as I was saying, it doesn't work if you call up help from the options page by clicking on a question mark icon: help then opens in a tab (as when IENG is updated) at the relevant heading level rather than a window, and as the options window remains in the foreground, it hides the help, which I find annoying. You then have to move the options window to read the help, or close the options window and then reopen it to change an option, which isn't very practical, is it? I found a workaround by deleting "modal" on line 1593 of the mboximport.js file but, as I'm not a developer, I'm not sure that's right! However, it works: the options window doesn't stay in the foreground and you can consult the help comfortably without having to close the options window.

@DenB10

kiki-ja commented 1 year ago

@DenB10

I understand what you are saying. Perhaps the addition of a debug option to open in a window, similar to PrintingTools NG, would help.

In the case of my PrintingTools NG: Since my PC display is not high resolution (FHD:1902x1080), I modified the width of the PTNG options window to be a little smaller and the PTNG help window in the same way so that both windows can be edited side by side.

sample

@kiki-ja

DenB10 commented 1 year ago

@kiki-ja

Perhaps the addition of a debug option to open in a window, similar to PrintingTools NG, would help.

You're teaching me something: I didn't know there was a debug option to open help in a window in PTNG, what is this way?

cleidigh commented 1 year ago

@DenB10 @kiki-ja Opening the options as non modal is indeed not a problem, it's the unintended consequences. For instance the thing is I would have to add detection of the options already open and prevent someone opening another dialog. I have not imagined other things, but there may be others.

As @kiki-ja mentioned, there is a debug option in PTNG at his suggestion to enable opening help in a separate window instead of a tab. I didn't get it into the UI and should have for v3.0.1 The debug option is helpinwin which you place in Debug Options

Short term I can do a preference for this then do the UI and translations in the next release. The question is what is the best way going forward and can we or should we do this before releasing v14.0.1? @cleidigh

DenB10 commented 1 year ago

@cleidigh

Opening the options as non modal is indeed not a problem, it's the unintended consequences. For instance the thing is I would have to add detection of the options already open and prevent someone opening another dialog. >Ihave not imagined other things, but there may be others.

In fact, I don't understand why IETNG's options window should be modal when PTNG's isn't!

As @kiki-ja mentioned, there is a debug option in PTNG at his suggestion to enable opening help in a separate window instead >of a tab. I didn't get it into the UI and should have for v3.0.1 The debug option is helpinwin which you place in Debug Options

"I must have shit in my eyes" (French expression) because this debug option is mentioned in the PTNG help. Anyway, thanks for the info.

Short term I can do a preference for this then do the UI and translations in the next release. The question is what is the best way going forward and can we or should we do this before releasing v14.0.1?

Don't worry, this is not a major concern and can wait. You've already done a lot of work on version 14.0.1 and I think (like you, I believe) it's time to release it.

PS: Did you read my question in my last PR comment about updating the French help?

Yarny0 commented 1 year ago

For the record: I tested all export methods/formats with a POP3/maildir and with a IMAP/maildir account. I didn't look into each exported file, but verified that their file size was approximately what was to be expected. Everything worked flawlessly. I also tested how export works with folded/unfolded messages and with only some messages selected; this also worked flawlessly: I always exported what I expected it to export, and I could no longer reproduce the problem where it effectively exported nothing.

At the end of my experiments, I again had the mbox-4-items-export-menu in the context menu of my (maildir) inbox folders, but again I found no way to reliably reproduce it.

Thanks for your work!

cleidigh commented 1 year ago

@DenB10 So I figured out the way to check if an option window is open and then realized I can avoid opening a second window if a user attempts AND then I focus on the already open window. Let me zip a pre7 and you can try to break it. Give me five... Yes I missed removing the export remote entry in help. @cleidigh

cleidigh commented 1 year ago

@DenB10 Ok here is the zip, you can install like an xpi. So if this doesn't break things we can do it. I may just do the window option as well. I need to delay to look at profile import export anyway. import-export-tools-ng-14.0.1-b7pre1-tb.zip

@cleidigh

cleidigh commented 1 year ago

@librelegal Excellent, thank you. The new selection handling should be correct for all scenarios now. @cleidigh

cleidigh commented 1 year ago

@Yarny0 Great, the selection handling should be spot on now. The menu issue is perplexing, I cannot figure that one out yet.. Btw just doing last minute change to make options non modal to view help better. See up thread if interested. @cleidigh

DenB10 commented 1 year ago

@cleidigh Wow, what responsiveness! After a few tries, I've had no problems whatsoever. You're a boss!

cleidigh commented 1 year ago

@DenB10 Cool, so far I can't think of anything other than preventing duplicate windows so hopefully we are good. For the window option I need the translation for: "Open help in a window instead of a tab" TheBoss

DenB10 commented 1 year ago

Pour le Boss : "Open help in a window instead of a tab" > "Ouvrir l’aide dans une fenêtre et non un onglet"

cleidigh commented 1 year ago

@DenB10 Merci mon frere Q: have you done anything with profile import export? Have a user I think is trying to import old one. Don't think I handle well in ui information. That's my next investigation. @cleidigh

DenB10 commented 1 year ago

@cleidigh

Q: have you done anything with profile import export?

In fact, I don't use IETNG's profile import-export function. I always use the same profile, which I manually save on a regular basis. If necessary, I restore a previous backup, taking care to clear Thunderbird's cache. I don't need several profiles either.

However, I've just done a few quick tests and it seems to work, except for a few quirks I can't explain: when I import a previously exported profile, traces of the previous profile seem to remain, even after restarting Thunderbird and clearing its cache. Maybe I'll tell you more tomorrow if you ask me to investigate further. But then again, I'm not really comfortable with this IETNG feature.

Edit: I must have been tired yesterday because I wrote some nonsense. I've tried importing and exporting profiles again and it works very well. This feature could be very interesting for those who handle several profiles. In any case, the weirdness I observed yesterday was due to the fact that Betterbird (which I use instead of Thunderbird) was still in memory when I opened the saved profile.

cleidigh commented 1 year ago

@Yarny0 While working on the help stuff I realized I had an error in the preference I added for you. The preference was:

extensions.importexporttoolsng.export.help.showOnInstallAndUpdate

should be, and will change for the release as of b7

extensions.importexporttoolsng.help.showOnInstallAndUpdate

Hopefully this is not to inconvenient, but better to change now. @cleidigh

sembuk commented 1 year ago

I usually use Thunderbird with 4 windows opened. "Select the destination directory" dialog opens 4 times when I exporting emails by selecting target directory and clicking save button. This behaviour happens also on importing, opening "Options" from right-click menu. My specs:

Windows 10 Pro 22H2
Thunderbird 115.4.2 (64-bit)
ImportExportTools NG v14.0.1-b7

cleidigh commented 1 year ago

@sembuk Thank you for catching this! Never using multiple windows I failed to contemplate or test this. I just posted b8 with the patch. Both the export and import, really all commands were affected. I tested with multiple windows, both the export and import. Hopefully you have a moment to verify. I would like to post to the review Q tomorrow. Thanks again @cleidigh

466

librelegal commented 1 year ago

@librelegal Excellent, thank you. The new selection handling should be correct for all scenarios now. @cleidigh

the save message to desktop function works in b8, thank you so much

cleidigh commented 1 year ago

@librelegal Thank you as always. I am posting v14.0.1 to the review Q today! This was certainly a big maintenance release %-} @cleidigh

sembuk commented 1 year ago

@sembuk Thank you for catching this! Never using multiple windows I failed to contemplate or test this. I just posted b8 with the patch. Both the export and import, really all commands were affected. I tested with multiple windows, both the export and import. Hopefully you have a moment to verify. I would like to post to the review Q tomorrow. Thanks again @cleidigh

v14.0.1 Release Status - Beta Download #466

Great, Works great, thank you for fixing this.

thunderbird / import-export-tools-ng

v14.0.1 Release Status - Beta Download #466

451 , #463

458

459

466

466