Empty filters are created if Editor is not shown

[Sungod3000]

Hi,

im noticing now that some of my filters are created completely empty. Im using TB 78 and im creating the filtern like i usually do but roughly half of them dont contain any trigger nor actions. The bug fixer in the filter window also recognizes those filters as empty and recommends to delete them. Any idea what would cause this?

Cheers

[RealRaven2000]

No idea - Can you elaborate on what you do to create the filter? Do you use the assistant + drag email method? Which of the templates / presets do you choose?

One thing I changed was the removal of own email addresses in the "from" template which used to be done silently and could lead to a filter without search terms - which effectively couldn't be edited. This will now generate a warning message and cancel filter creation. It would be great if you could come up with and document a reproduceable test case so I can investigate this in detail.

[Sungod3000]

Yes, I use the drag to filter method and I usually use the "based on domain" template. I get no error message, just the "filter created" notification.

I see if I can nail down when this happens to reproduce it.

[RealRaven2000]

Have you enabled the option "Open Editor after Filter Creation"?

[Sungod3000]

No, that I have disabled. I just created some new ones and checked individually that they were correctly created.

[RealRaven2000]

Ok, that may be the course of the problem. I will do some testing, with the editor it certainly works for me:

[Sungod3000]

Ah, and now that I check one again, its empty. That was maybe 5min after I created it. I just changed the language and restarted TB once.

However, the other filters that I combined with existing ones still work after the restart.

and i made some more filters and restarted after each, but they seems to be fine as well.

cant put my finger on it, ill keep looking.

[RealRaven2000]

I ran two more tests, one with the filter rules (editor) disabled and one with that and "display filters list" disabled - I think that's the case you are using as this will generated the "filte xx generated" message as only feedback. But in all cases both the search terms and action were added as expected. In the later case I see a warning:

Could not retrieve bundle string: qf.notify.filterCreated
Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIStringBundle.GetStringFromName]

but this is handled by the code gracefully without impacting functionality.

[RealRaven2000]

Ah, and now that I check one again, its empty. That was maybe 5min after I created it. I just changed the language and restarted TB once.

However, the other filters that I combined with existing ones still work after the restart.

and i made some more filters and restarted after each, but they seems to be fine as well.

cant put my finger on it, ill keep looking.

I think maybe the restart (without showing the filters list) may be the key here. I think by closing the filters list Thunderbird stores the filter to msgFilterRules.dat, that's probably a missing step. I am not sure how to "Make Thunderbird store the filters" works exactly but I will look into it... ~

(AFAIK the storage is handled by C++ code and cannot be directly scripted from JS)

[ThiloteE]

Just for the record: I also encountered this in late 2021, but it has been a while since it showed up. Currently i don't have any problems anymore.

Back then, I was not able to reliably reproduce the faulty filters, but they looked like this:

[aspensmonster]

I was able to reliably reproduce this issue. It looks like the failure scenario is when the Open Editor after filter creation is de-selected, and your activity ends up creating a new filter. The filter will appear to have been created. You can view it in the filter list. However, upon closing and restarting Thunderbird, the filter appears empty:

A reproduction of this behaviour is below:

https://user-images.githubusercontent.com/943591/179637502-936f24b2-0818-49f8-89a5-dfbf3d250074.mp4

However, when the Open Editor after filter creation option is selected, and your activity creates a new filter, the only way to get it to persist (as expected) is to click "OK" on the editor modal that opens up. Once that "OK" is clicked, the filter persists after a restart. See reproduction below:

https://user-images.githubusercontent.com/943591/179638316-f6c2fe89-7f05-43fe-8366-76d72d38fa2f.mp4

And, once the created filter is persisted, I am then able to turn off the Open Editor after filter creation option and streamline adding entries to the filter without needing to open the filter editor or filter lists up; such updates persist after restarts, as shown below:

https://user-images.githubusercontent.com/943591/179638656-1cfda03d-93d8-4cf0-8400-06fdd6f3890c.mp4

A few different work-arounds are:

1. Leave the Open Editor after filter creation option checked all the time. When a filter needs creating, you'll be forced to hit the OK button on the editor modal and save it. However, you'll also be forced to explicitly hit the OK button for every addition/merge with existing filters.
2. If you get a dialog about needing to create a new filter (i.e., there is no existing filter to merge with), and you have Open Editor after filter creation off, then there are a few options:
   1. Hit the Cancel button. This'll cancel creation of a filter. However, the email you were dragging still gets moved to its folder. Meaning you'd need to then move the email back where it was, and then pursue one of the below work-arounds.
   2. Tick the Filter Rules checkbox under Next Steps when you get the Create Filters modal from quickfilters, then click OK in the filter editor modal to ensure it gets persisted across restarts. However, this action results in the overall Open Editor after filter creation option to get re-checked until you either (1) un-tick it in quickFilters settings, or (2) un-tick it the next time the Create Filters modal from quickfilters is launched.
   3. When the Create Filters modal from quickFilters launches, and you note that it is creating a new filter, leave the Filter Rules checkbox under Next Steps un-checked and just click OK. Then, remember to open that newly-created filter in the normal Message Filters area, and then click OK to save it. This is of course error-prone and easy to forget.

Until a patch is in, I'm settling on 2.2. By default, keep the Open Editor after filter creation checkbox un-ticked. If I get a "need to create new filter" alert in the quickFilters Create Filters modal, then I'll (try) to remember to tick the Filter Rules checkbox under Next Steps first. And then the next time I'm updating an existing filter, I'll de-select that Filter Rules checkbox.

Kinda a hassle, but hey! At least there are re-production steps now :)

[aspensmonster]

Also, my TB version is 102.0.2 (64 bit) on GNU/Linux. I strongly suspect that issue #9 is probably the same issue, and less strongly suspect that issue #78 might also be (at least tangentially) related (though I'm not as sure there).

[RealRaven2000]

Thanks for providing this level of detail and posting the videos.

• Leave the Open Editor after filter creation option checked all the time. When a filter needs creating, you'll be forced to hit the OK button on the editor modal and save it. However, you'll also be forced to explicitly hit the OK button for every addition/merge with existing filters.

So if there was separate options for create vs. merge (and you could show the editor only when generating new filters) would this also considered an improvement?

1. Hit the Cancel button. This'll cancel creation of a filter. However, the email you were dragging still gets moved to its folder. Meaning you'd need to then move the email back where it was, and then pursue one of the below work-arounds.

...not necessarily, if you already have moved an email to a target folder there is no need to move it back in order to invoke the filter assistant. Instead use the context menu on the mail and select "quickFilters: Create filter from message..." - this also works with the merge logic:

2. Tick the Filter Rules checkbox under Next Steps when you get the Create Filters modal from quickfilters, then click OK in the filter editor modal to ensure it gets persisted across restarts. However, this action results in the overall Open Editor after filter creation option to get re-checked until you either (1) un-tick it in quickFilters settings, or (2) un-tick it the next time the Create Filters modal from quickfilters is launched.

So the reason for showing Filter Rules in the standard workflow is so you can run the filter manually after modification (to test or tidy up your inbox). But also saving the modified filters is done by the list dialog iirc, I need to look into how I can trigger persisting saving the modified filters. There is no direct command for saving filters, in the complicated function quickFilters.Worker.buildFilter() I am using filtersList.insertFilterAt(0, targetFilter); to add the filter to the passed in list. If the editor is displayed, hitting cancel deletes the filter again.

3. When the Create Filters modal from quickFilters launches, and you note that it is creating a new filter, leave the Filter Rules checkbox under Next Steps un-checked and just click OK. Then, remember to open that newly-created filter in the normal Message Filters area, and then click OK to save it. This is of course error-prone and easy to forget.

so one question here is - if the editor is shown but the list isn't, is then the filter also not saved (even if you click OK) ?

By the way I noticed you were using the "tools" menu to show the filter list, when you have a dedicated button on your toolbar:

[RealRaven2000]

Doing a fresh round of tests - ok, in my first test I created a new filter without showing the filters list (but showing the editor). It looks like the filter is added to the list, yet when I repeat the steps of generating another filter in the same folder, the merge step is not invoked. This means a fresh filters list is created when the assistant is opened. Weirdly, when I open the filters list, the new filter is actually present. So we dealing with 2 different filter lists.

technical notes for myself...

There seems to be a problem with finding a matching folder when we use the context menu right now - the filterAction (which is used to find a match) is not properly initialized - I think we would have to use heuristics to guess what the action is (move, copy or set flag). if we are in Inbox then most likely it won't be move or copy (create filter based on message assumes the action has been already carried out and deducts from that what the filter is meant to do); if it outside of inbox we can assume move for now (but the added complication could be an additional tag, so really it should be a list of actions).

Here is the call that leaves the filterAction undetermined:

  onMenuItemCommand: function onMenuItemCommand(e, cmd) {
   switch(cmd) {
      case 'toggle_Filters':
        quickFilters.Worker.toggle_FilterMode(!quickFilters.Util.AssistantActive); 
        break;
      case 'createFilterFromMsg':
          // ...
          // ...
          quickFilters.Worker.createFilterAsync_New(null, currentMessageFolder, messageList, null, false);

the 4th parameter of createFilterAsync_New() is the filterAction - it's null in this case. Guess I need to preinitialize it properly or deal with it better in createFilterAsync_New.

[RealRaven2000]

Dealing with the missing merge trigger on context menu first (because it is kind of annoying):

quickFilters-wx-5.6pre14.zip

I will do more testing in the afternoon

[RealRaven2000]

Dealing with the missing merge trigger on context menu first (because it is kind of annoying):

quickFilters-wx-5.6pre14.zip

I will do more testing in the afternoon

Look like I got stuck there and gave up temporarily... I possibly need to start from scratch...