search

RealRaven2000 / FiltaQuilla

Adds many new mail filter actions to Thunderbird
http://quickfilters.quickfolders.org/filtaquilla.html
GNU General Public License v3.0
88 stars 17 forks source link

options go missing when deselecting apply filter when "Getting new mail" #143

Open ioweb-gr opened 2 years ago

ioweb-gr commented 2 years ago

I try to create a new filter, but the regex options go missing when deselecting the option to apply when getting new mail

https://user-images.githubusercontent.com/20220341/147209581-83e988a9-f6eb-47db-bbcf-cadee579ebdc.mp4

RealRaven2000 commented 2 years ago

Regression? which version of THunderbird?

ioweb-gr commented 2 years ago

91.4.1 (64-bit) on Windows 11

RealRaven2000 commented 2 years ago

I tested it here (on Windows 7, which should behave the same)

image

And it doesn't remove the option when I remove the [x] Getting new Mail option (even after restarting the dialog). Do you get errors when unchecking in JavaScript error console? (Ctrl+Shift+J)

ioweb-gr commented 2 years ago

Not sure if anything is related to filtaquilla 

Unexpected event profile-after-change URLQueryStrippingListService.jsm:224
Unknown Collection "thunderbird/query-stripping" RemoteSettingsClient.jsm:160
1641043573165   tbsortfolders.folderPane    DEBUG   Init
1641043573165   tbsortfolders.folderPane    DEBUG   Add observer
xpunge - messenger - onLoad - activatedWhileWindowOpen: true
xpunge - xpunge_ti_onWindowLoad - Params Before: {"activatedWhileWindowOpen":true,"xpunge_ti_window_counter":-1,"getWindowCounter":0}
xpunge - xpunge_ti_onWindowLoad - Params After: {"activatedWhileWindowOpen":true,"xpunge_ti_window_counter":1,"getWindowCounter":1}
xpunge - xpunge_ti_onWindowLoad (on Window #1): Sat Jan 01 2022 15:26:13 GMT+0200 (Χειμερινή ώρα Ανατολικής Ευρώπης)

Opened Window #1

Checking Singleton Conditions ...
This Is The First Window. Setting Up Relative Timer ...

xpunge - xpunge_ti_setupStartup (on Window #1): Sat Jan 01 2022 15:26:13 GMT+0200 (Χειμερινή ώρα Ανατολικής Ευρώπης)

Relative Timer Checkbox NOT Selected.
Starting Timer Listener ...

xpunge - xpunge_ti_setupListener (on Window #1): Sat Jan 01 2022 15:26:13 GMT+0200 (Χειμερινή ώρα Ανατολικής Ευρώπης)

Started New Listener:
Listener ID = 5
Listener Interval = 1 minutes

WebExtensions: 91.4.1 qFi-messenger.js:15
WebExtensions: keynav.enableKeyNavigation: successfully enabled key navigation api.js:48
Trying to load C:\Program Files\Mozilla Thunderbird\libotr.dll OTRLib.jsm:64:11
Successfully loaded OTR library C:\Program Files\Mozilla Thunderbird\libotr.dll OTRLib.jsm:72:13
Successfully loaded OpenPGP library rnp.dll version 0.15.2+git20210806.dd923a4e.MZLA from C:\Program Files\Mozilla Thunderbird\rnp.dll RNPLib.jsm:92:15
Found 0 public keys and 0 secret keys (0 protected, 0 unprotected) RNPLib.jsm:288:15
tb.account.size_on_disk - Trying to set an unsigned scalar to a negative number. 16
tb.account.size_on_disk - Truncating float/double number. 2
tb.account.size_on_disk - Trying to set an unsigned scalar to a negative number. 6
tb.account.size_on_disk - Truncating float/double number. 2
tb.account.size_on_disk - Trying to set an unsigned scalar to a negative number. 18
tb.account.size_on_disk - Attempted to set the scalar to an incompatible value. 2
tb.account.size_on_disk - Trying to set an unsigned scalar to a negative number. 2
FiltaQuilla [logTime init]
 AddonManager: FiltaQuilla extension's version is 3.2 filtaquilla-util.js:267:13
services.settings: thunderbird/hijack-blocklists has signature disabled RemoteSettingsClient.jsm:1027
NS_ERROR_NOT_AVAILABLE: ActivityManager.jsm:129
Key event not available on some keyboard layouts: key=“h” modifiers=“accel,alt” id=“” messenger.xhtml
Key event not available on some keyboard layouts: key=“n” modifiers=“accel,alt” id=“” messenger.xhtml
Key event not available on some keyboard layouts: key=“b” modifiers=“accel,alt” id=“” messenger.xhtml
NS_ERROR_NOT_AVAILABLE: ActivityManager.jsm:129
LoginRecipes: getRecipes: falling back to a synchronous message for: imap://imap.gmail.com:993 LoginRecipes.jsm:295:9
Unknown Collection "thunderbird/password-recipes" RemoteSettingsClient.jsm:160
TypeError: browser is null
BrowserElementParent.jsm:24:21
TypeError: win.windowRoot is null
LoginManagerChild.jsm:2147:7
Some cookies are misusing the recommended “SameSite“ attribute 6
TypeError: browser is null
BrowserElementParent.jsm:24:21
TypeError: win.windowRoot is null
LoginManagerChild.jsm:2147:7
NS_ERROR_NOT_AVAILABLE: ActivityManager.jsm:129
TypeError: currentMsg is undefined
    showMsgDetails chrome://conversations/content/stub.bundle.js:1035
stub.bundle.js:1064:17
TypeError: currentMsg is undefined
    showMsgDetails chrome://conversations/content/stub.bundle.js:1035
stub.bundle.js:1064:17
NS_ERROR_NOT_AVAILABLE: 
FiltaQuilla [logTime init]
 fq_FilterEditor.js - start... filtaquilla-util.js:267:13
FiltaQuilla 2:18:40.832  [1 ms]   
 fq_FilterEditor.js - Finished. filtaquilla-util.js:267:13
FiltaQuilla 2:18:40.902  [70 ms]   
 attribute changed: filtaquilla-util.js:267:13
FiltaQuilla 2:18:40.903  [1 ms]   
 attribute changed: filtaquilla-util.js:267:13
FiltaQuilla 2:18:48.78  [7175 ms]   
 attribute changed: filtaquilla@mesquilla.com#headerRegex filtaquilla-util.js:267:13
Element.releaseCapture() is deprecated. Use Element.releasePointerCapture() instead. For more help https://developer.mozilla.org/docs/Web/API/Element/releasePointerCapture menupopup.js:163:13
FiltaQuilla 2:18:48.79  [1 ms]   
 Mutation observer (attribute), check for patching: [object XULElement] filtaquilla-util.js:267:13
mutation observer patched: [object XULElement] fq_FilterEditor.js:752:25
FiltaQuilla 2:18:48.80  [1 ms]   
 attribute changed: filtaquilla@mesquilla.com#headerRegex filtaquilla-util.js:267:13
FiltaQuilla 2:18:48.80  [0 ms]   
 Mutation observer (attribute), check for patching: [object XULElement] filtaquilla-util.js:267:13
FiltaQuilla 2:18:48.80  [0 ms]   
 attribute changed: filtaquilla@mesquilla.com#headerRegex filtaquilla-util.js:267:13
FiltaQuilla 2:18:48.80  [0 ms]   
 Mutation observer (attribute), check for patching: [object XULElement] filtaquilla-util.js:267:13
FiltaQuilla 2:18:48.81  [1 ms]   
 attribute changed: filtaquilla@mesquilla.com#headerRegex filtaquilla-util.js:267:13
FiltaQuilla 2:18:48.81  [0 ms]   
 Mutation observer (attribute), check for patching: [object XULElement]
RealRaven2000 commented 2 years ago

I only saw that you attached a video (not just a screenshot) - although I cannot reproduce the issue here I see that you had the Filtaquilla entries twice - which only happens if you reload the Add-on or change options (add / remove search conditions in Filtquilla settings) - Thunderbird has no good method of removing old custom options. Could the fact that you edited shown options in the same Thunderbird session be related to this issue? Make sure you restart Thunderbird before you test again,,,

RealRaven2000 commented 2 years ago

The only observable change I detect when creating a new filter and toggling the option "#Getting new mail":

image

ioweb-gr commented 2 years ago

I have restarted thunderbird many times since then and the issue persists. I'm not sure what's causing it. On filtaquilla settings I just checked the options for regexp choices. Nothing more, and only after noticing the issue. However I've moved my thunderbird profile because I recently reinstalled my OS, could that be a reason?

ioweb-gr commented 2 years ago

Is there somewhere where I can clear all filtaquilla settings to default?

RealRaven2000 commented 2 years ago

You could delete the preferences specific to FQ in prefs.js (while Thunderbird is closed) but I don't think it will make any difference. That would be all lines starting with

user_pref("extensions.filtaquilla

Note that any filters using the settings you removed won't work until you re-enable them on the options dialog (and these filters cannot be edited in a meaningful way unless their custom conditions are enabled)

Notwithstanding resetting these - do you always get duplication of fq choices in the condition dropdowns in a fresh Thunderbird session? PS: I am working with Filtaquilla 3.2 and see no such behavior in Tb 91.4.1 (Windows 10)

ioweb-gr commented 2 years ago

No I don't always see duplication. In new filters the same issue happens but no duplication. Have a look at this screen capture

https://user-images.githubusercontent.com/20220341/147872334-2d120106-f7b5-4be5-a3f6-01a7dcbb7b19.mp4

ioweb-gr commented 2 years ago

Does the data in prefs.js look normal?

    Line 71: user_pref("devtools.aboutdebugging.tmpExtDirPath", "C:\\Users\\gabtz\\Downloads\\FiltaQuilla-thunderbird68.zip");
    Line 132: user_pref("extensions.filtaquilla.HeaderRegexEnabled", true);
    Line 133: user_pref("extensions.filtaquilla.SubjectBodyRegexEnabled", true);
    Line 134: user_pref("extensions.filtaquilla.firstRun", false);
    Line 135: user_pref("extensions.filtaquilla.installDate", "2019-12-19");
    Line 136: user_pref("extensions.filtaquilla.markUnread.enabled", true);
    Line 137: user_pref("extensions.filtaquilla.removeKeyword.enabled", true);
    Line 138: user_pref("extensions.filtaquilla.trainAsGood.enabled", true);
    Line 139: user_pref("extensions.filtaquilla.version", "3.2");
RealRaven2000 commented 2 years ago

Does the data in prefs.js look normal?

  Line 71: user_pref("devtools.aboutdebugging.tmpExtDirPath", "C:\\Users\\gabtz\\Downloads\\FiltaQuilla-thunderbird68.zip");
  Line 132: user_pref("extensions.filtaquilla.HeaderRegexEnabled", true);
  Line 133: user_pref("extensions.filtaquilla.SubjectBodyRegexEnabled", true);
  Line 134: user_pref("extensions.filtaquilla.firstRun", false);
  Line 135: user_pref("extensions.filtaquilla.installDate", "2019-12-19");
  Line 136: user_pref("extensions.filtaquilla.markUnread.enabled", true);
  Line 137: user_pref("extensions.filtaquilla.removeKeyword.enabled", true);
  Line 138: user_pref("extensions.filtaquilla.trainAsGood.enabled", true);
  Line 139: user_pref("extensions.filtaquilla.version", "3.2");

yes - but can you check the tab tools / developer tools / debug addons

is anything listed in the section "temporary extensions"? After starting Thunderbird this should always read (0)

image

RealRaven2000 commented 2 years ago

what other ADd-ons are active?

ioweb-gr commented 2 years ago

image

image

image

RealRaven2000 commented 2 years ago

Can you try and disable "quicker quicksearch"? It's from the same author as "expression-search-ng" and he had a bug in there that affects correct parsing of the filters (breaks the function getAvailableOperators() which affects other custom search options like the ones added by FiltaQuilla), for which I submitted the following issue:

https://github.com/opto/Expression-Search-NG/issues/27

I would have also submitted a patch but his source code on github was out of date when I raised this. If disabling "quicker quicksearch" (maybe also restart Thunderbird just in case) resolves the issue I suggest you check the error console and raise a bug there.

RealRaven2000 commented 2 years ago

I checked the source code of "quicker quicksearch" but it doesn't seem to have this code, I may be wrong. Another one you might try to disable is "Search Results sort by Date" - basically anything to do with searching might be a candidate as it might also have code affecting the filter editor.

ioweb-gr commented 2 years ago

With all extensions disabled except filtaquilla the issue remains

image

image

RealRaven2000 commented 2 years ago

Maybe it's a restriction of the search scope - the methods are only shown if the database contains the data, which are these following cases:

  function _isLocalSearch(aSearchScope) {
    switch (aSearchScope) {
      case Ci.nsMsgSearchScope.offlineMail:
      case Ci.nsMsgSearchScope.offlineMailFilter:
      case Ci.nsMsgSearchScope.onlineMailFilter:
      case Ci.nsMsgSearchScope.localNews:
        return true;
      default:
        return false;
    }
  }

In all other scopes many of the search terms are hidden. The complete list of scopes is declared here: https://searchfox.org/comm-esr91/source/mailnews/search/public/nsMsgSearchCore.idl#18

  const nsMsgSearchScopeValue offlineMail = 0;
  const nsMsgSearchScopeValue offlineMailFilter = 1;
  const nsMsgSearchScopeValue onlineMail = 2;
  const nsMsgSearchScopeValue onlineMailFilter = 3;
  /// offline news, base table, no body or junk
  const nsMsgSearchScopeValue localNews = 4;
  const nsMsgSearchScopeValue news = 5;
  const nsMsgSearchScopeValue newsEx = 6;
  const nsMsgSearchScopeValue LDAP = 7;
  const nsMsgSearchScopeValue LocalAB = 8;
  const nsMsgSearchScopeValue allSearchableGroups = 9;
  const nsMsgSearchScopeValue newsFilter = 10;
  const nsMsgSearchScopeValue LocalABAnd = 11;
  const nsMsgSearchScopeValue LDAPAnd = 12;
  // IMAP and NEWS, searched using local headers
  const nsMsgSearchScopeValue onlineManual = 13;
  /// local news + junk
  const nsMsgSearchScopeValue localNewsJunk = 14;
  /// local news + body
  const nsMsgSearchScopeValue localNewsBody = 15;
  /// local news + junk + body
  const nsMsgSearchScopeValue localNewsJunkBody = 16;

So maybe you are trying to create the filter in a newsgroup? Or some in other way special account? Also if it is simply an IMAP account try to enable the Message Synchronizing Account setting:

image

ioweb-gr commented 2 years ago

Sorry for the long delay I missed this reply for some reason it was sent to spam.

The account is a gmail imap account and sync is enabled.

image

Would it be possible to log the context?

Moreover in the older accounts where I used the Header Regex filter, it's working normally but it's greyed out. So I can't edit/ modify the old filters but they work fine.

If I save the filter with "Getting New Email" checked, then try to edit and remove the option I get an error

image

So I was wondering if I could somehow modify the extension to list what it thinks the current context is

jenda69 commented 2 months ago

I'm having the same issue - FiltaQuilla options go missing / or grey out when unchecking Getting New Mail.

I have three mail accounts in my Thunderbird and this happens for both Gmail accounts. When creating a rule for the third IMAP account (on Vivaldi.net), I can see these options even when Getting New Mail is unchecked.

RealRaven2000 commented 2 months ago

I'm having the same issue - FiltaQuilla options go missing / or grey out when unchecking Getting New Mail.

I have three mail accounts in my Thunderbird and this happens for both Gmail accounts. When creating a rule for the third IMAP account (on Vivaldi.net), I can see these options even when Getting New Mail is unchecked.

can you check whether they have the same sunchronization + storage settings?

image

jenda69 commented 2 months ago

Huh, I overlooked the post about that. 🤦‍♂️ It wasn't checked. Now I checked it and the options no longer disappear. Thank you!

RealRaven2000 commented 2 months ago

Huh, I overlooked the post about that. 🤦‍♂️ It wasn't checked. Now I checked it and the options no longer disappear. Thank you!

thanks for reporting back. I think this is one of the filter conditions that has a dependency of certain search scopes where the match is valid - these restricitions are from the original author (Kent James), so I guess they are necessary. They may not actually be necessary for searching the subject line though, I could imagine that being downloaded as part of the header (?). Always hard to say what Thunderbird "actually" does with IMAP mail.

  // is this search scope local, and therefore valid for db-based terms?
  function _isLocalSearch(aSearchScope) {
    switch (aSearchScope) {
      case Ci.nsMsgSearchScope.offlineMail:
      case Ci.nsMsgSearchScope.offlineMailFilter:
      case Ci.nsMsgSearchScope.onlineMailFilter:
      case Ci.nsMsgSearchScope.localNews:
      case Ci.nsMsgSearchScope.newsFilter:
        return true;
      default:
        FiltaQuilla.Util.logDebug("isLocalSearch = FALSE!", aSearchScope);  // test!!!
        return false; 
    }
  }

// from implementation of self.bodyRegex 
    self.bodyRegex =
    {
      id: "filtaquilla@mesquilla.com#bodyRegex",
      name: util.getBundleString("fq.bodyRegex"),
      getAvailable: function bodyRegEx_getAvailable(scope, op) {
        if (scope == Ci.nsMsgSearchScope.newsFilter) return false;
        return _isLocalSearch(scope) && BodyRegexEnabled;
      },
      getAvailableOperators: function bodyRegEx_getAvailableOperators(scope) {
        if (!_isLocalSearch(scope))
        {
          return [];
        }
        return [Matches, DoesntMatch];
      },