Allow using escaped commas `\,` within text parameters of variables, where possible

[RealRaven2000]

Some variables such as the following replacement commands allow entering a piece of plain text for searching or replacing. (find, replace, altText. The problem is if the user wants to include a comma within the text - the parameters are broken up using a standard split routine, which regards the comma as delimiter between the params:

  %deleteText("text")%
  %replaceText("find","replace")%
  %matchTextFromBody("Regular Expression",MatchGroup,"altText")%
  %matchTextFromSubject("Regular Expression",MatchGroup,"altText")%
  %deleteQuotedText("searchText",quoteLevel)%
  %replaceQuotedText("searchText","replacementHTML",quoteLevel)%
  %preheader("headerText",className,"inlineStyles")%
  %file(imagePath,altText,attributeList)%
  %toclipboard(text)%
  %conditionalText(forwardMode,"inlineText","attachText")%
  %header.set(hdr,"insertText")%
  %header.prefix(hdr,"insertText")%
  %header.append(hdr,"insertText")%
  %header.delete(hdr,"insertText")%

So if we were to write an example like:

%replaceText("dogs","labrador, poodle and dalmation")%

this will be split into the following array with 3 parameters:

params = ['"dogs"', '"labrador', ' poodle and dalmation"']

Which will lead to undesired outcomes or the command failing completely. So in order to support commas within these text parameters, SmartTemplates should support escaped commas, like so:

%replaceText("dogs","labrador\, poodle and dalmation")%

The backslash will be used to tell the parser that the comma is part of a single parameter - the algorithm will detect these and recombine the array:

// PSEUDO CODE
// initial results (comma gets consumed as delimiter)
params = ['"dogs"', '"labrador\\', ' poodle and dalmation"']; 
// algorithm detects the trailing '\' and recombines the parameters, rebuilding the desired string
params = ['"dogs"', '"labrador, poodle and dalmation"']; 

[RealRaven2000]

Notes: %matchTextFromBody% and %matchTextFromSubject% and all %header.*()% commands already work with (unescaped) commas, as they have robust parsing mechanisms - they are looking ahead until they hit ") which includes everything else in the string.

Here is a new version which allows a bunch of new things:

smartTemplate-fx-4.4pre53.zip

my test file:

%toclipboard("pigs\, dogs\, birds and beetles")%
%header.set(subject,"here is some text for the subject\, a lot of text \, followed by clipped text: [",clipboard,"]")%

%file("../img/invalid-tab.png","here are some examples. 1\, 2\, 3...",width=450px,height=90px)%
<p>conditional text:
  %conditionalText(forwardMode,"here is some text\, inline","here is some text\, attachText")%
</p>

%replaceText("find this\, peasant","replaced\, adapted and overcome!")%
%deleteText("yours\,")%
<p>
  %matchTextFromSubject("some text\,12 not found",0,"nothing found, therefore I insert: Hello\, world!! Here\, there and everywhere.")%
</p>

%deleteQuotedText("blabla\, bla",1)%
%replaceQuotedText("search me, yeah","<p>hallo\, quoted world!</p>",1)%

---

To test the version above, download the zip file, drag the file into Thunderbird Add-ons Manager, do not extract contents or if won't install.

[RealRaven2000]

Next version, this one allows (escaped) commas in multiparameter variables such as %replaceQuotedText()% and %replaceText()%.

Examples on how to escape all commas:

%replaceText("find this\, peasant","replaced\, adapted and overcome!")%
%replaceQuotedText("search me\, yeah","<p>hallo\, quoted world\, and every one else!</p>",1)%
%header.set(subject,"here is some text for the subject\, a lot of text \, followed by clipped text: [",clipboard,"]")%

note: only %header()% commands allow multiple parameters like in the example above, which includes mixing literal strings with clipboard.

smartTemplate-fx-4.4pre61.zip

---

To test the version above, download the zip file, drag the file into Thunderbird Add-ons Manager, do not extract contents or if won't install.

[RealRaven2000]

While testing with smart fragments I found both the function %replaceText()% and %deleteText()% quite useless because they only try to replace text within the template, which usually doesn't contain any matching text, so at least for this case I decided to iterate all childNodes of the <body> element instead, excluding any <blockquote> elements. (for that we have the deleteQuotedText() and replaceQuotedText() functions)

This means we can now modify an email we have just written (or loaded via a template) by replacing text via a smart fragment, which makes it actually potentially useful.

Here is the result: smartTemplate-fx-4.4pre66.zip

---

To test the version above, download the zip file, drag the file into Thunderbird Add-ons Manager, do not extract contents or if won't install.

[RealRaven2000]

Implemented in 4.4 - published 28/03/2024