Extension debugging from running filters in Thunderbird

[diabolusss]

Hello! I'm sorry if this it not the right place where to ask for a help, but as you are maintaining this add-on i suppose you have th right skills to assist me. I've added case-insensitive regex flag into code, like this:

` function _getRegEx(aSearchValue) {
    /*
     * If there are no flags added, you can add a regex expression without
     * / delimiters. If we detect a / though, we will look for flags and
     * add them to the regex search. See bug m165.
     */
    let searchValue = aSearchValue,
        searchFlags = "";
    if (aSearchValue.charAt(0) == "/") {
      let lastSlashIndex = aSearchValue.lastIndexOf("/");
      searchValue = aSearchValue.substring(1, lastSlashIndex);
      searchFlags = aSearchValue.substring(lastSlashIndex + 1);
    }
    if(regexpCaseInsensitiveEnabled && !searchFlags.contains("i")){
      searchFlags+="i";
    }
    dump("dump");
     Components.utils.reportError("error"); 
    util.logToConsole("_getRegEx log");
    util.logError("_getRegEx err", "src", "1", "2", "3", "");
   // util.logException("_getRegEx ex" );
    console.log("console");

    return [searchValue, searchFlags];
  }`

I've added it to options and can change its value from there, but this is not the topic of this question. When i run filter, this flag is not used. Thus, i've added logging into the function, but i see no o/p in any console Thunderbird has, specifically Error console and from developer tab (i suppose it's the same). On the other hand, if i add same logging into " self._init = function() " then logs appears (just from init) after add-on install.

Please, show me the proper way to debug this function or where to search logs from this place or what's from with flag...

[RealRaven2000]

Try the js error console (tools / dev tools / error console) and make sure to enable these 4 options at least:

[diabolusss]

Thanks for the tip, i've found a problem... It turns out that it's not enough to reinstall add-on, thunderbird must be restarted, too. After that everything is working as expected including logs and case-insensitive regex by default.

 Services.console.logStringMessage("serv: " + searchFlags); //working
   dump("dump"); //not working
   Components.utils.reportError("error"); //working
   util.logToConsole("_getRegEx log");//working
   util.logError("_getRegEx err", "src", "1", "2", "3", "");//working
   console.log("console");//working

[RealRaven2000]

Thanks for the tip, i've found a problem... It turns out that it's not enough to reinstall add-on, thunderbird must be restarted, too. After that everything is working as expected including logs and case-insensitive regex by default.

the problem is that for actions there is no remove method in Thunderbird so I am afraid, yes - currently you can't add a new method without restarting. I spent many weeks on converting this but that was one of the things I couldn't solve.

Do you want me to merge the code or can you create a PR?

[diabolusss]

It's not the big deal to restart it, it's just annoying. I've installed restart add-on for that.

Do you want me to merge the code or can you create a PR?

Thank you, i'll push it to git when it will be ready or if you want to try the code i can do that already. Body regex is firing, just main function doesn't get real result.

[RealRaven2000]

Hello! I'm sorry if this it not the right place where to ask for a help, but as you are maintaining this add-on i suppose you have th right skills to assist me. I've added case-insensitive regex flag into code, like this:

` function _getRegEx(aSearchValue) {
    /*
     * If there are no flags added, you can add a regex expression without
     * / delimiters. If we detect a / though, we will look for flags and
     * add them to the regex search. See bug m165.
     */
    let searchValue = aSearchValue,
        searchFlags = "";
    if (aSearchValue.charAt(0) == "/") {
      let lastSlashIndex = aSearchValue.lastIndexOf("/");
      searchValue = aSearchValue.substring(1, lastSlashIndex);
      searchFlags = aSearchValue.substring(lastSlashIndex + 1);
    }
    if(regexpCaseInsensitiveEnabled && !searchFlags.contains("i")){
      searchFlags+="i";
    }
    dump("dump");
     Components.utils.reportError("error"); 
    util.logToConsole("_getRegEx log");
    util.logError("_getRegEx err", "src", "1", "2", "3", "");
   // util.logException("_getRegEx ex" );
    console.log("console");

    return [searchValue, searchFlags];
  }`

just looking at the existing function - doesn't that already support adding /i as a flag in the regular expression? Why add it as a global flag as well?

[diabolusss]

I think that all regexp filters should work in case agnostic mode by default, but to override that, I added an option in settings. Basically when you are adding some filters, and it can be a lot of them, it's, first, hard to maintain them, and, secondly, I believe that it's redundant information to add it to every rule. I mean, it's easier to enable an option in settings than for, say, 20 rules to add backslashes with custom flag.

[diabolusss]

@RealRaven2000 hi, i have a bug with new filter clauses. Basically, filtering is working fine using your workaround with calling alert to allow callbackObject to get processed. I'm wondering is there a way to at least replace this annoying alert to smth that don't need user interaction? I've tried async/await, but haven't succeeded at that...

When editing filter, my new clause input fields are missing. Values are stored totally fine, just fields are not visible. To make them visible, i must choose some existing clause from the list. After that, it stays visible until i reopen this window. https://imgur.com/OVko3DO.png

[RealRaven2000]

@RealRaven2000 hi, i have a bug with new filter clauses. Basically, filtering is working fine using your workaround with calling alert to allow callbackObject to get processed. I'm wondering is there a way to at least replace this annoying alert to smth that don't need user interaction? I've tried async/await, but haven't succeeded at that...

Where did I say you need to call alert? Is this something from Kent James' original documentation?

[RealRaven2000]

I think that all regexp filters should work in case agnostic mode by default, but to override that, I added an option in settings. Basically when you are adding some filters, and it can be a lot of them, it's, first, hard to maintain them, and, secondly, I believe that it's redundant information to add it to every rule. I mean, it's easier to enable an option in settings than for, say, 20 rules to add backslashes with custom flag.

how would you disable it then? Do we need to build UI for it....

[diabolusss]

@RealRaven2000 hi, i have a bug with new filter clauses. Basically, filtering is working fine using your workaround with calling alert to allow callbackObject to get processed. I'm wondering is there a way to at least replace this annoying alert to smth that don't need user interaction? I've tried async/await, but haven't succeeded at that...

Where did I say you need to call alert? Is this something from Kent James' original documentation?

Ok, sorry, my mistake. It's in code, so i've assumed it's yours... Anyway, see filtaquilla.js#L1125. Like i've said It allows body to be fetched and processed.

I think that all regexp filters should work in case agnostic mode by default, but to override that, I added an option in settings. Basically when you are adding some filters, and it can be a lot of them, it's, first, hard to maintain them, and, secondly, I believe that it's redundant information to add it to every rule. I mean, it's easier to enable an option in settings than for, say, 20 rules to add backslashes with custom flag.

how would you disable it then? Do we need to build UI for it....

Well, i've added a custom flag "c" which actually exists in tcl (see tcl ?c flag). And, of course, it's in effect only if the according setting in the plugin is enabled.

[RealRaven2000]

how would you disable it then? Do we need to build UI for it....

Well, i've added a custom flag "c" which actually exists in tcl (see tcl ?c flag). And, of course, it's in effect only if the according setting in the plugin is enabled.

What's "Tcl"? That doesn't look like a JavaScript regular expression flag. /i or /c maybe but I have never seen (?c)

[diabolusss]

how would you disable it then? Do we need to build UI for it....

Well, i've added a custom flag "c" which actually exists in tcl (see tcl ?c flag). And, of course, it's in effect only if the according setting in the plugin is enabled.

What's "Tcl"? That doesn't look like a JavaScript regular expression flag. /i or /c maybe but I have never seen (?c)

Of course, it's used here as a normal JavaScript regular expression flag in a form: /expression/c.

[diabolusss]

When editing filter, my new clause input fields are missing. Values are stored totally fine, just fields are not visible. To make them visible, i must choose some existing clause from the list. After that, it stays visible until i reopen this window. https://imgur.com/OVko3DO.png

Do you have any clue what can cause this bug?

[RealRaven2000]

When editing filter, my new clause input fields are missing. Values are stored totally fine, just fields are not visible. To make them visible, i must choose some existing clause from the list. After that, it stays visible until i reopen this window. https://imgur.com/OVko3DO.png

Do you have any clue what can cause this bug?

Yes I had this before with the JavaScript field. I believe it's caused by expanding of quote marks (or other syntax) which makes the html parser fail - you need to store the code in an interim variable and then copy into the textbox.

[diabolusss]

But it's a standard regex field, like Subject regex. And It's missing always when opening this window (or creating new filter clause) - even for empty input or basic one, like one word without special chars and regex flag delimiters. That's strange. And i suppose, if it was a html parser error, then there should be errors in console, and it should fail always.

Debug log, when opening filter rules editor for filter with subjectBodyRegex and bodyRegex:

 FiltaQuilla [logTime init]
fq_FilterEditor.js - start...
11:48:50.828 FiltaQuilla 11:48:50.828  [1 ms]   
fq_FilterEditor.js - Finished.
11:48:51.292 FiltaQuilla 11:48:51.292  [464 ms]   
Mutation observer (childList), check for patching: [object XULElement]
11:48:51.296 FiltaQuilla 11:48:51.296  [4 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.296 FiltaQuilla 11:48:51.296  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.296 FiltaQuilla 11:48:51.296  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.296 FiltaQuilla 11:48:51.296  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.296 FiltaQuilla 11:48:51.296  [0 ms]   
Mutation observer (childList), check for patching: [object XULElement]
11:48:51.298 FiltaQuilla 11:48:51.298  [2 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.298 FiltaQuilla 11:48:51.298  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.298 FiltaQuilla 11:48:51.298  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.298 FiltaQuilla 11:48:51.298  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.299 FiltaQuilla 11:48:51.299  [1 ms]   
Mutation observer (childList), check for patching: [object XULElement]
11:48:51.301 FiltaQuilla 11:48:51.301  [2 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.301 FiltaQuilla 11:48:51.301  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.301 FiltaQuilla 11:48:51.301  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.301 FiltaQuilla 11:48:51.301  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.301 FiltaQuilla 11:48:51.301  [0 ms]   
Mutation observer (childList), check for patching: [object XULElement]
11:48:51.303 FiltaQuilla 11:48:51.303  [2 ms]   
attribute changed: filtaquilla@mesquilla.com#bodyRegex
11:48:51.303 FiltaQuilla 11:48:51.303  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.303 FiltaQuilla 11:48:51.303  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#bodyRegex
11:48:51.304 FiltaQuilla 11:48:51.303  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.304 FiltaQuilla 11:48:51.304  [1 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#subjectBodyRegex
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#bodyRegex
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
attribute changed: filtaquilla@mesquilla.com#bodyRegex
11:48:51.304 FiltaQuilla 11:48:51.304  [0 ms]   
Mutation observer (attribute), check for patching: [object XULElement]

​Body regex has simple rule: "intel|i7|i5|i3", and subjectBodyRegex has: "Price:\s([1-5][0-9][0-9]|[1-9][0-9]|[0-9])\s€" (without quotes).

[RealRaven2000]

When editing filter, my new clause input fields are missing. Values are stored totally fine, just fields are not visible. To make them visible, i must choose some existing clause from the list. After that, it stays visible until i reopen this window. https://imgur.com/OVko3DO.png

Do you have any clue what can cause this bug?

I committed a patch. you forgot to modify fq_FilterEditor.js. This is not a standard text field so it needed to be handled specially. The filter editor is hell anyway.

[diabolusss]

When editing filter, my new clause input fields are missing. Values are stored totally fine, just fields are not visible. To make them visible, i must choose some existing clause from the list. After that, it stays visible until i reopen this window. https://imgur.com/OVko3DO.png

Do you have any clue what can cause this bug?

I committed a patch. you forgot to modify fq_FilterEditor.js. This is not a standard text field so it needed to be handled specially. The filter editor is hell anyway.

Thanks.

[diabolusss]

Hi @RealRaven2000 , i've made a pull request https://github.com/diabolusss/FiltaQuilla/pull/1. For easier regexp callback function maintain i've extracted those into utils file. I believe it could be useful, but pull request have some conflicts i can't resolve.

[RealRaven2000]

Functionality was added in release 3.2