Make quickFilters compatible with Thunderbird 78 ESR

[RealRaven2000]

Thunderbird 78 will reject classic Add-ons. For quickFilters to be supported, it must be rewritten as a "mail extension experiment" which is the preparation for next years full on "Mail extension" conversion where all XUL and XPCOM functions have to be replaced with HTML and native API functions (many of which still need to be implemented in the THunderbird core team).

At this moment it is unclear which special restrictions the conversion to "real" mail extensions is going to bring with it - I am liaising with the Thunderbird development team to get as much functionality supported as possible, for next year.

[RealRaven2000]

Me and @opto are currently working on porting QuickFolders to Tb78, quickFilters will follow shortly after.

[RealRaven2000]

Timeline: we intend to be ready by end of September, when 78.2 is auto-rolled out to the Tb68 users base. Please do not upgrade until then.

What happened - @jobisoft helped me and @opto with the first hard conversion step to a mail experiment with QuickFolders. We will finish that one first so that it runs completely with only minor functionality loss. Then we will apply the same changes to quickFilters (similar codebase) and last SmartTemplate⁴.

[RealRaven2000]

making progress with QuickFolders conversion - got some help from @jobisoft yesterday and made a fresh branch at https://github.com/RealRaven2000/QuickFolders/tree/ESR78-method

[jorchier]

Please notify me of all changes. Thanks ...

[RealRaven2000]

Please notify me of all changes. Thanks ...

Will do! Any activity here will promptly land in your inbox. If you're interested in the ongoing struggle, QuickFolders is the first conversion candidate, see the relevant issue over there. Once done there I will apply the shared code changes to quickFilters and fix the remaining stuff here.

[RealRaven2000]

Still working on the QuickFolders conversion, which is going to my the first fully Tb78 compatible Add-on, I have just created a new code branch to do preparation work for converting quickFilters:

https://github.com/RealRaven2000/quickFilters/tree/ESR78

For conversion steps here, we will be following the new upgrade guide here:

https://github.com/thundernest/addon-developer-support/wiki/Using-the-WindowListener-API-to-convert-a-Legacy-Overlay-WebExtension-into-a-MailExtension-for-Thunderbird-78

Since we will still be using the old XPCOM functions from within the confines of the "Mail extension experiment" framework, we hope to not lose any functionality in this step. The next 12 months can then be used to replace these functions with APIs (some of which still have to be designed and implemented by the Thunderbird Core team) as they become available. There will be separate issues logged for this work.

Thunderbird has also planned a big amount of work removing the old interfaces, see: https://developer.thunderbird.net/planning/roadmap no doubt this may impact on the ESR period for Thunderbird 78; we will deal with the problems caused for us as they occur as usual.

[EpicOrange]

Thank you for your work! posting to watch

[RealRaven2000]

I know there is a lot of interest in this work - we are not quite there yet, still ironing out kinks in QuickFolders - but we will have a version for quickFilters very soon.

For those interested / or also using it, a solid beta for QuickFolders in Tb78 is here.

[alexgogan]

Hi Axel,

Really (REALLY) looking forward to getting this update soon, new pc installed latest version and hundreds of filters cant import >:¬{ oh my inbox!

If I can help with something just ask.

[franzschoeller]

For all who are waiting for update so QF can be used with 78.xx - on 30th september i got information that pre-release shall be available in week 41. So let's hope and see. :)

BR

[RealRaven2000]

Here is the first test version:

https://github.com/RealRaven2000/quickFilters/releases/tag/5.0pre37

please add all bugs here in the thread below... ⬇⬇⬇

[leonewland]

Installed quickFilters 5.0pre37 on iMac running Catalina 10.15.6 OS with Thunderbird 78.3.1 and quickFilters successfully alphabetized my 200 or so filters (my reason for adding quickFilters). However, now my "Move to" function for messages does not work.

So far, only bug I've found is the inactivation of the 'move to' function for messages.

My only suggestion for 5.0pre37 is that if you add a new filter to an alphabetized list, it should put it in its proper alphabetical place rather than just random.

[RealRaven2000]

Installed quickFilters 5.0pre37 on iMac running Catalina 10.15.6 OS with Thunderbird 78.3.1 and quickFilters successfully alphabetized my 200 or so filters (my reason for adding quickFilters). However, now my "Move to" function for messages does not work.

So far, only bug I've found is the inactivation of the 'move to' function for messages.

It sounds more like you have one "broken" filter or a filter that specifies "stop running filters" which inhibits the other filters further down the list.

My only suggestion for 5.0pre37 is that if you add a new filter to an alphabetized list, it should put it in its proper alphabetical place rather than just random.

well, or you could just activate this very option on the New Filter Properties tab:

[x] insert new filter in alphabetical order

this works best if your filter list is already alphabetical.

Tip: The quickest way to open quickFilters options is by _right-clickin_g the filter-assistant button:

[ranman3g]

Hi Axel, Just loaded qF 5.0pre39 with TB 78.3.1 Great to have qF back in the fold.

Only issue I have found so far is the "find filters" option doesn't find any filters for a folder even though I know they are there.

Thanks so much for all the quick/hard work getting this back.

[RealRaven2000]

Hi Axel, Just loaded qF 5.0pre39 with TB 78.3.1 Great to have qF back in the fold.

Only issue I have found so far is the "find filters" option doesn't find any filters for a folder even though I know they are there.

Thanks so much for all the quick/hard work getting this back.

which find filters command exactly? if you use the "built in" search box you should be able to find it:

• First use the cogwheel / search setting to enable searching for filter actions

• then it should work (in this example I am searching for a filter that moves mail to the folder "avaaz" - I renamed it to something dissimilar for testing):

Is that the way you referred to when you wrote "find filters"?

Also as an aside, the cannonical initialism for quickFilters is qF. (QF is for QuickFolders) :)

[ranman3g]

Raven,
The "find filters" I was referring to was the one that occurs when you right click a folder:

This used to pop up the message filters window with the filter being referenced in focus. Now it just says no filter found.

Good to know about the search options dropdown n the message filters box. I don't typically have the message filters window up though unless I already am modifying or adding a filter.

And good to know about the qf nomenclature....doh!

[RealRaven2000]

Raven, The "find filters" I was referring to was the one that occurs when you right click a folder: (image snipped) This used to pop up the message filters window with the filter being referenced in focus. Now it just says no filter found.

Ok, found it! Good to know somebody actually uses this feature - it's a selling point for licenses so that's a good thing. 🤑

I hope I got it right - the latest version for testing can be found here:

https://github.com/RealRaven2000/quickFilters/releases/tag/5.0pre46

This one uses the target branch ESR78 which is going to be used for all future development from Thunderbird 78 onwards.

[ranman3g]

In 5.0pre46 the right click "find filters" option now opens the message filters window if it is not open and sometimes puts the focus on the right filter. Most of the time however it does not find the right filter and instead simply places the focus on the first filter in the list for the Local Folders account.

Also if the message filters window is open and you right click the "find filters" option, the message filters window closes.

[RealRaven2000]

In 5.0pre46 the right click "find filters" option now opens the message filters window if it is not open and sometimes puts the focus on the right filter. Most of the time however it does not find the right filter and instead simply places the focus on the first filter in the list for the Local Folders account.

I will look into it - there is a "search results" dropdown in the top portion for the message filters, try selecting an entry there - it may select the correct account for you.

Note that the search process has to go across account boundaries (I always move my mails from my pop3 gmail inbox across to a different account that is "local only" - a remnant from an older email address I used to use before their ISP closed it down. I don't really use the "Local Folders" account, that's why I may not have encountered it.

Also if the message filters window is open and you right click the "find filters" option, the message filters window closes. ok - fixing this one should be easy enough.

[RealRaven2000]

@ranman3g - I have uploaded a new prerelease here, version 5.0pre48 - can you test it please? It should address and possibly fix the "find filter" problems.

[franzschoeller]

Hi, i tried to install, but does not work - "file seems to be corrupted". BR

[RealRaven2000]

Hi, i tried to install, but does not work - "file seems to be corrupted". BR

Oh I am sorry to hear that - I think @opto added a file that is needed and it wasn't part of the build script.

[RealRaven2000]

Hi, i tried to install, but does not work - "file seems to be corrupted". BR

Fixed it - update popup is now fully internationalisable (Only English + German have been translated so far, everything else will default to English)

https://github.com/RealRaven2000/quickFilters/releases/tag/5.0pre56

[franzschoeller]

Thanks a lot, works fine!

[ranman3g]

@ranman3g - I have uploaded a new prerelease here, version 5.0pre48 - can you test it please? It should address and possibly fix the "find filter" problems.

Hi Raven, 5.0pre48 is flagged as corrupt and will not install. BTW, TB pushed 78.3.2 out just now and that is what I am running.

I tried 5.0pre56 and it still won't find the filters.

I also have a somewhat complicated folder structure. I have a number of POP3 and IMAP email accounts with folder structures below them and then for legacy reasons I have two primary POP3 accounts that are configured to channel into "Local Folders".

My filters list has been built over many years to support the folder structure so some filters are based on the incoming email accounts (username@example1.com, generalmail@example2.com, etc.) and many are based on "Local Folders". All were created by hand up until my discovery of qf,

My understanding is the application of the filters is sequential....i.e. Filters associated with incoming email address specific accounts are applied prior to any associated with "Local Folders".

I experimented a bit and found that:

After rebooting TB, if the message filters window is not open, the "find filters" button will open it with no account selected but will display the filters for "Local Folders" and will highlight the first filter in that list which is irrelevant to the filter being searched.

Once I have selected an account to search in the message filter window, the default account always is "Local Folders".
As an example, I searched for the filters associated with my github folder:

The actual filter associated with github mail is associated with my incoming GeneralMail account, not with Local Folders.

Leaving the message window open, if I change the account to GeneralMail and then use the "find filters" button, it does find the filter:

If the message filters window is open, it seems to not allow the "find filters" function to change the account being searched.

If the message window is closed, the "find filters" function works if the filter is only associated with an incoming email account that does not end up in Local Folders. If the filter is associated with an incoming email account that is configured to deliver to Local Folders then the "find filters" function can't seem to get past the Local Folders account to search the filter list for the incoming email account.

Thanks for trying to get this to work. I have a large number of folders and keeping track of all the filters is a pain without this function.

[RealRaven2000]

@ranman3g can you check JavaScrpit error console for any clues? This is often the quickest way to find regression bugs like these.

[ranman3g]

Hi Raven, WRT 5.0pre48 not installing, I see the following error:

WRT the "find filter" function not changing the account, I see the following error/log:

[RealRaven2000]

Hi Raven, WRT 5.0pre48 not installing, I see the following error:

yeah, I know - that version is incomplete because @opto added a file to the root (i18n.js) that wasn't added to my build script. Please use 5.0pre56 instead! (The versions higher prerelease number always contains all changes from lower prerelease number)

WRT the "find filter" function not changing the account, I see the following error/log:

this one may be something to work if - is it from pre56?

[ranman3g]

this one may be something to work if - is it from pre56?

Yes, it was from pre56. Thanks

[RealRaven2000]

this one may be something to work if - is it from pre56?

Yes, it was from pre56. Thanks

@ranman3g - interestingly just before the XML parsing error there is a line referencing the file quickfolders-preferences.js - indicating you also have QuickFolders installed. Can you try reproduce it with QuickFolders disabled?

[RealRaven2000]

@ranman3g I have added a new version of QuickFolders that will fix the other error message (which may or may not lead to the issue you described) - if you like, you can install it from here:

https://github.com/RealRaven2000/QuickFolders/releases/tag/5.0pre100

[ranman3g]

@RealRaven,

Yes I do have QF loaded and I just now updated it to pre100 (Thanks for the link). I tried it with/without QF with TB reboots and the behavior is the same.

Error messages are gone but the issue still remains.  qf can't seem to change the account selected in the message filters window.

I see a log message but I think it would be expected:

On 10/13/2020 2:38 AM, RealRaven2000 wrote:

@ranman3g https://github.com/ranman3g I have added a new version of QuickFolders that will fix the other error message (which may or may not lead to the issue you described) - if you like, you can install it from here:

https://github.com/RealRaven2000/QuickFolders/releases/tag/5.0pre100 https://github.com/RealRaven2000/QuickFolders/releases/tag/5.0pre100

[RealRaven2000]

OK, let's go in a bit deeper - I am able to do this on my account and it will select the correct one provided there is a filter that moves mail there. Can you please enable a couple of debug flags:

1) open quickFilters options (via right-click on the filter assistant icon) and click advanced tab 2) on the advanced tab, activate [x] debug mode on the bottom right 3) rightclick the debug mode checkbox and (after dismissing the warning) toggle the following settings to true:

extensions.quickfilters.debug.filterSearch
extensions.quickfilters.debug.filterSearch.detail

this will give me additional lines in the JavaScript log, hopefully we will see an error or warning pop up in there

my expected log ends something like this (there is more detail, but I think it's the last few messages that matter):

quickFilters {FILTERSEARCH} 15:52:38.109  [0 ms]   
checking account [seawonkey@gmail.com] for target folder: imap://seawonkey%40gmail.com@mail.gmail.com/[Gmail]/Areneco
iterating 2 filters...
quickFilters {FILTERSEARCH} 15:52:38.109  [0 ms]   
FOUND FILTER MATCH:
[Gmail] � Areneco
quickFilters {FILTERSEARCH.DETAIL} 15:52:38.110  [1 ms]   
[0] Current Filter URI:imap://seawonkey%40gmail.com@mail.gmail.com/[Gmail]/Areneco
[0] Current Filter URI:mailbox://Realraven2000%40hotmail.com@pop3.live.com/Templates

quickFilters {FILTERSEARCH} 15:52:38.110  [0 ms]   
checking account [news.mozilla.org] for target folder: imap://seawonkey%40gmail.com@mail.gmail.com/[Gmail]/Areneco
iterating 0 filters...
quickFilters {FILTERSEARCH.DETAIL} 15:52:38.110  [0 ms]   

quickFilters {FILTERSEARCH} 15:52:38.110  [0 ms]   
Matches found: 1
quickFilters 15:52:38.110  [0 ms]   
findFromTargetFolder(Areneco) COMPLETE 
1 matches found

[ranman3g]

Hi Raven, The detail is quite voluminous and contains some personal info due to pathnames etc. Happy to send to you but not post to the world. Please advise how I can make it available to you. Thanks!

[RealRaven2000]

ranman3g - you can send it directly through the mail link at the bottom of my support web sites (mail to Axel Grude). See here:

https://quickfilters.quickfolders.org/version.html#contactMe

[ranman3g]

@RealRaven2000, Yesterday sent logs and data for both initial filter search and followup with the message filter window open and proper account selected. Interesting that the initial filter search (no message filter window open) appears to iterate at least twice. Second scenario once the proper account has been selected searches only once.

[JohnFLand]

FYI, quickFilters 5.0pre56 (as well as some earlier 5.0pre versions) conflicts with the Quick Folder Move extension (https://addons.thunderbird.net/en-US/thunderbird/addon/quick-folder-move/?src=userprofile) -- the QFM extension will pop up a menu with Shift-M and Shift-Y, but will not actually move or copy selected email(s) to the selected folder. Right now, that function is more important to me in my daily work, so unfortunately I've had to disable quickFilters (which I also like!).

[RealRaven2000]

FYI, quickFilters 5.0pre56 (as well as some earlier 5.0pre versions) conflicts with the Quick Folder Move extension (https://addons.thunderbird.net/en-US/thunderbird/addon/quick-folder-move/?src=userprofile) -- the QFM extension will pop up a menu with Shift-M and Shift-Y, but will not actually move or copy selected email(s) to the selected folder. Right now, that function is more important to me in my daily work, so unfortunately I've had to disable quickFilters (which I also like!).

Right, I guess you could raise a bug with its author? I have my own quickMove and quickJump functions in my own Add-on QuickFolders, but it's slightly different from Philip Kewisch's one. Mine also allows entering parent names and even creating new child folders when you move mail. You can use a slash to get at folders lower in the hierachy:

that's why I didn't see a need to install an Add-on from another person, but you gotta use what your used to.

[RealRaven2000]

The lastest betaversion can be downloaded from here:

https://github.com/RealRaven2000/quickFilters/releases/tag/5.0pre63

It addresses some license validation issues and adds full localisation (18 languages) to the splash screen, which also features the new Add-on Icon. This is also used more consistently in places like the Add-on Manager.

[RealRaven2000]

A new betaversion is available, which is addressing issue #33 - it fixes the move to / copy to context menus (at least on my Windows system)

https://github.com/RealRaven2000/quickFilters/releases/tag/5.0pre69

There will probably be one or two more beta releases before the final release early November.

[RealRaven2000]

Latest betaversion:

https://github.com/RealRaven2000/quickFilters/releases/tag/5.0pre71

Minor fixes (Assistant centered on screen, Japanese translations for splash page corrected by Kozuke Kaizuka)

[cghost161]

please inform me, when quickfilters is compatible with thunderbird 78.5 thanks karl blaas karl.blaas@magic.ms

[RealRaven2000]

Final release for Thunderbird 78, I just uploaded to ATN, once reviewed it will be available there: https://addons.thunderbird.net/thunderbird/addon/quickfilters/versions/?page=1#version-5.0

here on github you can always load from: https://github.com/RealRaven2000/quickFilters/releases/tag/5.00

[RealRaven2000]

I pushed a new emergency release today because of borked locales (also Tb78 regressions), see:

https://github.com/RealRaven2000/quickFilters/releases/tag/5.0.2

[rgrun1]

I had an issue with quick filters: I tried to teach/filter a bulk email to a folder. It suggested Mailing List but failed saying I needed to add "list-id" I already have "List-id" and have it successfully assigned in several filters. I added the lower case version and the next attempt caused a java script error. I tried another, "subject" I think and it was still asking for "list-id" There is no "list-id" in the message headers Thanks Rick

[RealRaven2000]

I had an issue with quick filters: I tried to teach/filter a bulk email to a folder. It suggested Mailing List but failed saying I needed to add "list-id" I already have "List-id" and have it successfully assigned in several filters. I added the lower case version and the next attempt caused a java script error. I tried another, "subject" I think and it was still asking for "list-id" There is no "list-id" in the message headers Thanks Rick

Could you create a new issue to track this one please?