"Forward with SmartTemplate" - implement triggering template from filter (using FiltaQuilla)

[RealRaven2000]

This would be very useful for mass-forwarding / processing of Emails.

As it is primarily tasked with adding automatic filter actions / conditions, FiltaQuilla would need to implement a new menu item "Forward with SmartTemplates" which would have to trigger smartTemplates, telling it which HTML file to use as template.

SmartTemplates will implement a listener on the background page so that it can get the command from FiltaQuilla to reply to a message with a certain template. The message header URI would have to be forwarded so that ST can identify the original message. It will then trigger a reply (through composer, because that's the only place ST can currently process mail), autopmatically open the email editer, process it as usual (e.g. setting a new recipient, from address, subject line etc.) and after completion automatically send off the message.

Corresponding FiltaQuilla issue: issue 153

[RealRaven2000]

A lot of the SmartTemplates specific detail is being discussed on the FQ side, see the following comments

https://github.com/RealRaven2000/FiltaQuilla/issues/153#issuecomment-1048670130 https://github.com/RealRaven2000/FiltaQuilla/issues/153#issuecomment-1048693113 https://github.com/RealRaven2000/FiltaQuilla/issues/153#issuecomment-1048716926 https://github.com/RealRaven2000/FiltaQuilla/issues/153#issuecomment-1048728403

As there are just examples they are not necessarily part of the specifications for the changes in SmartTemplates.

[RealRaven2000]

As I commented in the Filtaqilla issue, this cannot be set up to work together yet, until I have coded changes in both Add-ons - which may take a while as it is complex.

However you can already test the forward manually and prepare the template. For this you (1) write a HTML file that contains the commands set all recpients with the individual email addresses - using Cc (so that none of the recipients see the other recipients, if that's what you want. otherwise you can use "to" instead of "cc"):

%sig(none)%
%header.set(cc,"USER2.1")%
%header.append(cc,"USER2.2")%
%header.append(cc,"USER2.3")%
etc.

(2) In SmartTemplates settings, highlight "Forward" for the Server account, and configure as follows:

this will make sure the quote Header (what you call header) is not inserted before the forwarded part.

To test the html file , click on the dropdown beside forward and select the html template from step (1):

The (account specific) settings from step (2) configure remove the original quote header - this rule also applies when you use a HTML template.

Once we have new versions ready to test, the filter action will select the file instead!

[RafalGemex]

Axel, this is super urgent - do you know when will you be able to implement it, please?

[RealRaven2000]

Axel, this is super urgent - do you know when will you be able to implement it, please?

I am already working on some prototype code. It's very complex. I have programmed the front-end and new action selector in filtaQuilla today, and then the code that sends it to the background page of FQ and another that sends it on to the external Add-on (SmartTemplates). The message includes the template file name and a converted message header that is API compatible to uniquely identify the message. This is not trivial because the API compatible side (communicating via background scripts) have no concept of the way the internal back-end (which I deal with via legacy code) stores messages. Luckily I have solved this problem in the PrintingToolsNG support work, which we just tested successfully for the first time this week.

On the SmartTemplates side, I have so far implemented a receiver for the message from the filter action, so it definitely is aware that it needs to do something. I will still have to create the "meat" of the issue which triggers composer with the template and then automatically sends off the message. It will obviously fail if the template does not set any to / cc fields so we need to design a way of handling with this to avoid of potentially having 100s of open composer windows hanging around. One way would be to save the message as draft and close the window at this stage...

Preview version (for the filter UI): filtaquilla-3.4pre45.zip

I have also prepared UI for "respond with template) but I haven't tested that. Both new Actions must be enabled via FiltaQuilla preferences so they can show up in the filter editor:

SmartTemplates side, so far: smartTemplate-fx-3.11pre7.zip

This doesn't do anything obvious but it does print to the error console (Ctrl+Shift+J) :

SmartTemplates.fileTemplates.onExternalMessageRun() smartTemplate-fileTemplates.js:1279:13 SmartTemplates.BackgroundUpdate.forwardWithTemplate { target: ChromeWindow chrome://messenger/content/messenger.xhtml, isTrusted: true, srcElement: ChromeWindow chrome://messenger/content/messenger.xhtml, currentTarget: ChromeWindow chrome://messenger/content/messenger.xhtml, eventPhase: 2, bubbles: false, cancelable: false, returnValue: true, defaultPrevented: false, composed: false, … } undefined undefined smartTemplate-fileTemplates.js:1280:13 SmartTemplates forwarded message succesfully. st-background.js:247:21

[RealRaven2000]

Axel, this is super urgent - do you know when will you be able to implement it, please?

I am already working on some prototype code. It's very complex. I have programmed the front-end and new action selector in filtaQuilla today, and then the code that sends it to the background page of FQ and another that sends it on to the external Add-on (SmartTemplates). The message includes the template file name and a converted message header that is API compatible to uniquely identify the message. This is not trivial because the API compatible side (communicating via background scripts) have no concept of the way the internal back-end (which I deal with via legacy code) stores messages. Luckily I have solved this problem in the PrintingToolsNG support work, which we just tested successfully for the first time this week.

On the SmartTemplates side, I have so far implemented a receiver for the message from the filter action, so it definitely is aware that it needs to do something. I will still have to create the "meat" of the issue which triggers composer with the template and then automatically sends off the message. It will obviously fail if the template does not set any to / cc fields so we need to design a way of handling with this to avoid of potentially having 100s of open composer windows hanging around. One way would be to save the message as draft and close the window at this stage...

Preview version (for the filter UI): filtaquilla-3.4pre45.zip

I have also prepared UI for "respond with template) but I haven't tested that. Both new Actions must be enabled via FiltaQuilla preferences so they can show up in the filter editor:

SmartTemplates side, so far: smartTemplate-fx-3.11pre7.zip

This doesn't do anything obvious but it does print to the error console (Ctrl+Shift+J) :

SmartTemplates.fileTemplates.onExternalMessageRun() smartTemplate-fileTemplates.js:1279:13 SmartTemplates.BackgroundUpdate.forwardWithTemplate { target: ChromeWindow chrome://messenger/content/messenger.xhtml, isTrusted: true, srcElement: ChromeWindow chrome://messenger/content/messenger.xhtml, currentTarget: ChromeWindow chrome://messenger/content/messenger.xhtml, eventPhase: 2, bubbles: false, cancelable: false, returnValue: true, defaultPrevented: false, composed: false, … } undefined undefined smartTemplate-fileTemplates.js:1280:13 SmartTemplates forwarded message succesfully. st-background.js:247:21

[RealRaven2000]

@RafalGemex there is a new version at https://github.com/RealRaven2000/SmartTemplates/issues/174#issuecomment-1050905721

it contains the new SmartTemplates command %suppressQuoteHeaders% - for maximum flexibility this will always remove the quote header, independently of what is set in your account setting. So it is an override which you will find useful.

[RealRaven2000]

I have written some test code over the weekend as I know this feature is urgent for @RafalGemex - importantly, if you want to move the mail with through the same file (standard action: move mail to folder "X") then the composer will not initialize properly: the email is moved and Composer cannot set up the content of the forwarded mail in time. The workaround is to use the FiltaQuilla feature "Move Later" which delays the execution of moving the email by 5 seconds.

[RealRaven2000]

test versions: filtaquilla-3.4pre50.zip

smartTemplate-fx-3.11pre42.zip

(to install, download both zip files and drag them into Thunderbird Add-ons manager)

Test filter:

The template file forward-ag.html used to forward contained the following code

%sig(none)%
%suppressQuoteHeaders%
%header.set(to,"axel.grude@gmail.com")%
%header.append(to,"Real Raven <realraven2000@hotmail.com>")%
%header.append(to,"Seamonkey <seawonkey@gmail.com>")%
%header.delete(subject,"Fwd: ")%
%header.prefix(subject,"[SmartTemplates] FORWARDER - ")%

You can replace to with bcc if you do not want the recipients to see each other:

%header.set(bcc,"user 2.1")%
%header.append(bcc,"user 2.2")%
%header.append(bcc,"user 2.3")%

(where user 2.N represents an email address)

[RealRaven2000]

To view the email for a few seconds (after fully processing & before it is sent off automatically) you can increase the setting

extensions.smartTemplate4.fileTemplates.sendTimeout

in Thunderbird's Configuration Editor (the default is 200 [ms]). One thing that needs to be tested it the behavior when multiple emails arrive at exactly the same time; there might be a problem in that case if the filters fire much faster than the Composer Window can process initialisation commands from SmartTemplates. In order to trigger a template, the template name and mail URI are stored in a global variable before calling the internal Thunderbird function ComposeMessage( ).

At the heart of the processing specific to the cross-Add-on message from the filter is this function:

  // [issue 173] function to trigger mailing with a template from a filter (FiltaQuilla feature issue 153)
  onExternalMailProcess: function(data, composeType) {
    // similar to onItemClick / onSelectAdHoc
    console.log("SmartTemplates.fileTemplates.onExternalMailProcess()");
    console.log(composeType, data);
    let msgHeader = data.messageHeader;

    SmartTemplate4.fileTemplates.armedEntry = 
    { 
      composeType: composeType, 
      path: data.templateURL, 
      message: msgHeader,
      isAutoSend: true  // new flag 
    };
    // simulate a reply to this message!
    let realMessage = SmartTemplate4.Util.extension.messageManager.get(msgHeader.id),
        uri = realMessage.folder.getUriForMsg(realMessage);   
    SmartTemplate4.Util.logDebug("Sending SmartTemplate triggered by external Add-on", msgHeader);

    let msgUris = new Array(uri);
    let aCompType;
    switch(composeType) {
      case "fwd":
        aCompType = Ci.nsIMsgCompType.ForwardInline;
        break;
      case "rsp":
        // .ReplyAll  .ReplyToSender  .ReplyToGroup  .ReplyToSenderAndGroup  .ReplyToList:
        // what about special reply cases, like these?
        aCompType = Ci.nsIMsgCompType.Reply;
        break;
    }
    ComposeMessage(aCompType, Ci.nsIMsgCompFormat.Default, realMessage.folder, msgUris);
  },

There is also be a similar Filter Command "Reply with SmartTemplate" which might be of interest - this will quote the original mail as if clicking on a reply button - again you can remove the header with the %suppressQuoteHeaders% command if required. The main difference is that the original mail will be in a <blockquote> tag.

[RealRaven2000]

This version deals with messages being dropped if multiple mails come in that satisfy the filter criteria. Instead of using a global variable in the main window I am now adding the settings for auto-forwarding / auto-replying mails to an array which will be investigated for the correct item and copied in when Composer receives Thunderbird's NotifyComposeBodyReady event:

If the queue contains items they will be matched on original message id and type of composition (respond or forward)

    } else if(theQueue.length) {
      let origUri = gMsgCompose.originalMsgURI;
      // try to find matching item from the queue
      let found = theQueue.find(el => el.uri == origUri && el.composeType == getMsgComposetype());
      if (found) {
        theFileTemplate = found;
        theQueue = theQueue.filter(el => el != found);
      }
      else
        theFileTemplate = theQueue.pop(); // if we can't find, let's take the last item instead and hope for the best.
    }

smartTemplate-fx-3.11pre49.zip

to install the new version, as always download the zip find and drag into Thunderbird Add-ons Manager.

[RealRaven2000]

IMplemented in 3.11 published on 18/03/2022