Updating to Thunderbird 102 can generate invalid tabs

[RealRaven2000]

Updating to Thunderbird 102 can completely invalidate QuickFolders tabs by changing folder addresses!!!

I missed that the Thunderbird Core team has decided to combine settings realhostname / hostname so that the folders now need different folder URIs. It cannot know that Add-ons that may use these URIs for storing folder locations - here is the bug:

bug 1573690

The only way to fix this (apart from deleting and recreating the invalid tabs) currently is by exporting the QuickFolders settings (via QF license tab / save configuration), finding the invalid folders and editing the now faulty folder URIs: replacing the old root of the URI with the new root.

The root URI is built like this:

mail.server.server*.userName + "@" + mail.server.server*.hostname

The correct host + user name were stored in:

mail.server.server*.realuserName
mail.server.server*.realhostName

In Tb 102 the prefixed "real" values were used to overwrite the userName / hostName settings. (The old "alias" settings are purged from the prefs file so there is no way to find these from the settings without looking at a backup)

[RealRaven2000]

Another related bug:

https://bugzilla.mozilla.org/show_bug.cgi?id=1483485

[RealRaven2000]

Tasks: going forward store folder URIs as tuples of {accountKey, relative_path}

accountKey is used to deduct the correct account and from that build the root URI (the part mailbox://user@server/ or newsgroup://user@server/)

relative_path is the rest of the folder URI (the right hand side as seen relative to the root)

Advantage: once converted, this storage method is immune to fallout from renaming accounts / Thunderbird 102 simplifying the accounts. Problems: when transferring folders between 2 machines (via Save / Load Configuration) accountKeys can mismatch (the key is just an internal identifier, with a count that depends on in which order accounts were generated) - mitigating functions for this case need to be explored later.

Code Snippets from John:

// old:
let uri = folder.URI

// new: 
let account = MailServices.accounts.FindAccountForServer(folder.server).key
let uri = {account, path: folderURIToPath(account, folder.URI)}
or use a fake string : 
let uri = `${account}:/${folderURIToPath(account, folder.URI)}

See https://searchfox.org/comm-central/source/mail/components/extensions/parent/ext-mail.js#1600

... to get from account and path back to a valid URI (if you need to):

let URI = MailServices.accounts.getAccount(accountId).incomingServer.serverURI + path;

[tjukken]

How annoying. I guess I'll look into downgrading to 91 and never upgrade again.

Well, that was painless. I read one had to use command line switches. But I simply renamed the new profile folder, and copied the old one. Now QuickFolders works again!

[RealRaven2000]

I am working on a patch - the first part will be released today. All URIs will be transitioned (even for users of Tb91) to use a more resilient method of storing folder URIs. For the moment (5.10.1) I am just adding the account id for each tab, then reconstruction code to fix all URIs automatically will follow later. The idea is to move away from URI anyway as mail extensions do not use it either. they all use "tuples" of {Account + relative path}.

Folder URIs will only be constructed and used internally during the session for performance reasons but it shouldn't really be something my users should worry about / know.

[RealRaven2000]

After releasing hotpatch 5.10.1 I found another minor problem in the loadConfig function when it tries to visually update all tabs - these were bugs introduced by the conversion of the options dialog to HTML. There will be another hotpatch addressing this tomorrow. Here is a preview version for immediate testing:

QuickFolders-mx-5.10.2pre4.zip

To install this prerelease, download zip file and drag into Thunderbird Add-ons Manager (don't unpack)

[RealRaven2000]

...And here is a version that attempts to repair broken folders in Thunderbird 102 (but only if the account was stored in 91.* using 5.10.1 beforehand) by rebuilding the URI using the account attribute. Note that this may not work if you transfer a config file between 2 different profiles / computers as the same server may have a different account id. This is something that I am not sure that the Thunderbird team fully thought through when they made the patch - it will also affect backed up filters as they store absolute folder URIs as well.

There will be some log entries in JS error console when QuickFolders tries to correct the folder URIs.

QuickFolders-mx-5.10.2pre5.zip

To install this prerelease, download zip file and drag into Thunderbird Add-ons Manager (don't unpack)

[RealRaven2000]

Last version before release (includes some layout fixes and an additional debug menu item to copy expected account / URI info for any currently selected folder)

this copies the current account and folder URI to the clipboard (for support / fixing invalid tabs)

To get troubleshooting information on an invalid tab, right-click and select QuickFolders Command / Tab specific properties

this will copy the faulty URI and (guessed) account information for the tab into the clipboard, for example:

Invalid Tab:
QuickFolders Tab: [space test 2 BORKED URI]
account: account1
URI: imap://thunderbirddaily67%wef40gmail.com@imap.gmail.com/addons/quickfolders/space test 2 borked

New Version:

QuickFolders-mx-5.10.2pre12.zip

To install this prerelease, download zip file and drag into Thunderbird Add-ons Manager (don't unpack)

[rakoczy]

Would be nice if you could describe an appropriate procedure to check whether an incorrect label may exist and what can be a consequence of the relocation of messages when a tab points to an incorrect folder. Is it possible to guess where a moved message may land?

I assume that it is enough to select a tab in QF and see if the proper TH folder is shown in reaction. However, I would be happier knowing how to check it without QF enabled...

[RealRaven2000]

I just found out that the new repair routine (which can use a valid account id, if it was stored in Thunderbird 91) does not work for IMAP folders. I will release a patched version very shortly. I was shown a function to generate a new URI through the following code in QuickFolders.Model.reconstructUri():

  let oldUri = Services.io.newURI(oldURI)
  return ac.incomingServer.serverURI + oldUri.pathQueryRef; 

Unfortunately Services.io.newURI will only work for URIS beginning with mailbox:// (POP3) and news:// (newsgroups) but it throws an error for URIs starting with imap:// (unless the folder actually exists, which is exactly what won't be the case when we try to repair it).

[RealRaven2000]

Work completed in 5.10.3 released on 14/4/2022 - this is all I can do to try to fix the problem automatically.

Offical version on atn.

If you are already in Thunderbird 102 and have a bunch of invalid tabs, you can fix them by two methods:

1 - EITHER: deleting all invalid tabs (using QuickFolders tools / find orphaned tabs) and manually dragging the folders back to create fresh tabs. Only do this if you have only a handful of tabs!

2 - OR: To fix many tabs very quickly, you can export the configuration to json, edit the file and reimport the fixed file. For this you need to find out the old username@server portion of any invalid URI and replace with the correct new one.

this copies the current account and folder URI to the clipboard (for support / fixing invalid tabs):

Folder Inbox
On account1
URI: imap://thunderbirddaily67%40gmail.com@imap.gmail.com/INBOX

the username / server portion here is "thunderbirddaily67%40gmail.com"

To get troubleshooting information on an invalid tab, right-click and select QuickFolders Command / Tab specific properties

this will copy the faulty URI and (guessed) account information for the tab into the clipboard, for example:

Invalid Tab:
QuickFolders Tab: [space test 2 BORKED URI]
account: ?account
URI: imap://thunderbirddaily67%oldserver.xyz.com@imap.gmail.com/INBOX

the portion "thunderbirddaily67%oldserver.xyz.com@imap.gmail.com" is what needs to be replaced with the correct part above: "thunderbirddaily67%40gmail.com".

Once you do this with all the strings in config.json, you can reimport the tabs on the license tab in QF Settings:

activate the setting "include QuickFolders tabs" - you will get a preview of the imported tabs before committing to the results.

[tjukken]

Upgraded from 91 to 102 now. Tabs are correct again, but Thunderbird is super laggy. Disable QuickFolders and lag disappears.

[RealRaven2000]

Upgraded from 91 to 102 now. Tabs are correct again, but Thunderbird is super laggy. Disable QuickFolders and lag disappears.

This is known, and handled in issue #283 - there is a fresh version there in the latest comment: QuickFolders-mx-5.11pre77.zip

[tjukken]

Thank you, that fixed it.

[tmetro]

RealRaven2000 commented on Jul 17

you can export the configuration to json, edit the file and reimport the fixed file.

I followed this procedure, and it worked. The tabs were again functional. I used the procedure to correct folders that has been moved from a (now no longer supported) movemail account to a local folder account.

But after a restart of TB the problematic folders returned to being orphaned. Other folders residing on IMAP accounts remained working.

I disabled and then immediately enabled QuickFolders without restarting TB and the orphaned folders were again fixed.

I've seen this happen at least twice. Not every time TB is restarted are the folders orphaned, but sometimes they are.

[RealRaven2000]

you can export the configuration to json, edit the file and reimport the fixed file.

I followed this procedure, and it worked. The tabs were again functional. I used the procedure to correct folders that has been moved from a (now no longer supported) movemail account to a local folder account.

But after a restart of TB the problematic folders returned to being orphaned.

this is a strange one - I have one other user who has the problem with his Local Folders / Trash account. that mysteriously gets re-connected to a Trash mail folder on another account (presumably IMAP). I wonder if there is something with Local Folders causing them (sometimes) to not be ready for a while when starting up Thunderbird... You could try setting the startup delay on advanced settings to 0 to see whether it makes the problem happen more often: