Cardbook: Deferred variable (clicking on the pink "to(name)" field not pulling data from B intermittently

[RealRaven2000]

Sometimes the cardbook database does not return any date when clicking on the pink "to" field (deferred variables) after composre was opened from the Cardbook address book, using the context menu command "WRite a New Message" on an entry:

[richanl]

actual configuration is: betterbird115, smarttemplate4.1 and cardbook90.9. Result in error console while writing new message after clicking "TO"-field:

<window.controllers/Controllers sollte nicht mehr verwendet werden. Verwenden Sie es nicht für die Browser-Erkennung. 2 blank SmartTemplates st-composer.js - onLoad() smartTemplate-util.js:733:12 SmartTemplates smartTemplate-composer.js smartTemplate-util.js:733:12 Object { MessageHdr: "" } smartTemplate-composer.js:247:11 SmartTemplates added compose-window-init listener smartTemplate-util.js:733:12 SmartTemplates [logTime init] SmartTemplate4.init() smartTemplate-util.js:689:13 SmartTemplates 10:12:3.553 [0 ms]
SmartTemplate4.init() ends. smartTemplate-util.js:689:13 SmartTemplates 10:12:3.554 [1 ms]
SmartTemplate4.notifyComposeBodyReady() gMsgCompose type: 0 smartTemplate-util.js:689:13 SmartTemplates {IDENTITIES} 10:12:3.554 [0 ms]
Retrieved msgIdentity key value: id8 smartTemplate-util.js:683:13 SmartTemplates {IDENTITIES} 10:12:3.554 [0 ms]
Retrieved identity from msgIdentity.Identity key = id8 identityName = "xxxxxxxxxxxxxxxx" ra@xxxxxxxxxxxx smartTemplate-util.js:683:13 SmartTemplates 10:12:3.556 [2 ms]
checkIsURLencoded() Not an encoded string, this may be a SmartTemplates header: %to(addressbook,nickname)% smartTemplate-util.js:689:13 SmartTemplates ST.composer.load smartTemplate-util.js:733:12 SmartTemplates 10:12:3.556 [0 ms]
composer.initTemplateMenu() ... smartTemplate-util.js:689:13 SmartTemplates 10:12:3.557 [1 ms]
composer.initSnippetMenu() ... smartTemplate-util.js:689:13 SmartTemplates 10:12:3.558 [1 ms]
getBoolPref(extensions.smartTemplate4.header.inject) failed: Err:[Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIPrefBranch.getBoolPref]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: chrome://smarttemplate4/content/smartTemplate-prefs.js :: getBoolPref :: line 68" data: no] smartTemplate-util.js:689:13 SmartTemplates 10:12:3.559 [1 ms]
Version() getter. addonInfo: Object { id: "smarttemplate4@thunderbird.extension", name: "SmartTemplates", description: "Automatisiert das Erstellen von Antworten und Einfügen von verschiedenen Mail-Informationen. Unterstützt Vorlagen per Identität sowie externe HTML Dateien.", version: "4.1", mayDisable: true, enabled: true, optionsUrl: "", installType: "normal", type: "extension", permissions: (8) […], … } smartTemplate-util.js:689:13 SmartTemplates resetDocument(withUndo = true) smartTemplate-util.js:733:12 SmartTemplates 10:12:3.563 [4 ms]
Setting up listeners for deferred field variables! smartTemplate-util.js:689:13 SmartTemplates resetDocument(withUndo = true) smartTemplate-util.js:733:12 SmartTemplates {MIME} 10:12:5.398 [1835 ms]
mimeDecoder.detectCharset guessed charset: UTF-8... smartTemplate-util.js:683:13 SmartTemplates {MIME} 10:12:5.398 [0 ms]
mimeDecoder.detectCharset guessed charset: UTF-8... smartTemplate-util.js:683:13 SmartTemplates 10:12:5.398 [0 ms]
Correcting MIME encoded word from AB: =?UTF-8?Q?xxxxxxxxxxxxxx?= Info@xxxxxxxxx.de to:xxxxxxxxxxxxxxxxxx Info@xxxxxxxxxxxx.de Guessed charset: UTF-8 smartTemplate-util.js:689:13 SmartTemplates resolveDeferred(isReplaceField:false) called without nodeList

smartTemplate-util.js:739:11 SmartTemplates 10:12:5.405 [7 ms] Did you run template variables from smart snippets menu? Are you composing in plaintext mode? smartTemplate-util.js:689:13 >

[RealRaven2000]

I will make a version for 115 later that gives additional debug output...

[richanl]

On 19.09.2023, cardbook 90.2 was published. Since this release, I don't get the described errors (ST4.1 05.05.2023 and betterbird 115.2.0-bb10). On Write/answer/forward, SmartTemplates fetches %recipient(addressbook,nickname)% from cardbook properly.

[richanl]

One problem left: If mail-addresses in my Cardbook are stored with capitals as Hans.Mayer@, Smart template doesn't recognize it in the Mail with %recipient(addressbook,nickname)%. Do I change it to hans.mayer@, it works on my system. Same Problem with "answer": If the senders mail-address is transmitted with capitals, ST4 doesn't recognize it. The letters must be in lower case. Can you fix this behaviour?

[RealRaven2000]

One problem left: If mail-addresses in my Cardbook are stored with capitals as Hans.Mayer@, Smart template doesn't recognize it in the Mail with %recipient(addressbook,nickname)%. Do I change it to hans.mayer@, it works on my system. Same Problem with "answer": If the senders mail-address is transmitted with capitals, ST4 doesn't recognize it. The letters must be in lower case. Can you fix this behaviour?

I need to look into it - this may be a problem on the Cardbook / lookup side. Obviously I cannot know in which way Cardbook stores the contacts, so I assume it should always convert every email address to lowercase and search based on that.... I will ask Phillipe about that one.

[richanl]

cardbook has the ability to change addresses with the exclamation mark button:

But this doesn't help, if the sender I want to answer, has captials in the mail-header.

[RealRaven2000]

Can you check in error console, to see whether I request the email in lowercase or mixed case?

First enable the debug settings

extensions.smartTemplate4.debug.adressbook = true
extensions.smartTemplate4.debug.mime.split = true

(Just open advanced settings, activate debug mode, right-click debug mode)

this should tell us what email address it asks Cardbook for actually.

[richanl]

Here are the console-outputs of two adresses, one working, the other failing. console-export-2023-9-26_7-56-8.txt console-export-2023-9-26_7-51-34.txt

[richanl]

Further tests strengthen the suspicion that it is indeed the case in the mail address, which causes this error.

[RealRaven2000]

OK for the moment this is a bug on the cardbook side (it should really be agnostic to case, returning a match independently of what case the passed in email address is). As a workaround for now I can add a switch that forces conversion to lowercase when looking up the email, but this may also backfire if you store the address with mixed case. That's why I won't enable this switch by default. I will ask Phillippe (@cardbook) whether he can provide a quick fix in the next version of cardbook.

for this to work, of course you have to make sure that all your contacts are stored in lowercase:

note: this may catch out some users if they receive mixed case email and store that format in Cardbook without fixing it manually. That's why I still think Cardbook must be fixed to guard against this by allowing case insensitive lookup.

here is the workaround version:

smartTemplate-fx-4.4pre14.zip

---

to install, please download the zip file and drag it into Thunderbird Extension Manager (do not unpack contents it won't install that way)

[richanl]

DIN 5008 describes "rules for writing and layout of word and information processing" Paragraph 22.2. describes the form of email addresses: In fact there are no capital letters to be found. And yes, I find capitals only in mails that I received from German partners. All received international mail-addresses follow DIN 5008. It is important that "incorrect" mail addresses are processed by @cardbook and @RealRaven2000 in such a way that there are no incorrect results. The current workaround provides this functionality. The two programmers are probably better placed to judge whether this workaround is a clean solution from a software point of view. Cardbook offers a correction button that can automatically convert addresses to international spelling before saving them. This allows anyone to create a standardised display of new e-mail addresses:

[RealRaven2000]

It is important that "incorrect" mail addresses are processed by @CardBook and @RealRaven2000 in such a way that there are no incorrect results. The current workaround provides this functionality. The two programmers are probably better placed to judge whether this workaround is a clean solution from a software point of view. Cardbook offers a correction button that can automatically convert addresses to international spelling before saving them. This allows anyone to create a standardized display of new e-mail addresses:

As often, the usage of email addresses can vary and people can mistype addresses believing uppercase or even camelcase might be correct, so if I convert to lowercase or not, we need the cardbook side to retrieve the record (even if the user was wrong), even if the email is stored incorrectly.

[RealRaven2000]

I have reached out to the author of Cardbook and asked him to fix it in his gitlab:

https://gitlab.com/CardBook/CardBook/-/issues/1601

[RealRaven2000]

One problem left: If mail-addresses in my Cardbook are stored with capitals as Hans.Mayer@, Smart template doesn't recognize it in the Mail with %recipient(addressbook,nickname)%. Do I change it to hans.mayer@, it works on my system. Same Problem with "answer": If the senders mail-address is transmitted with capitals, ST4 doesn't recognize it. The letters must be in lower case. Can you fix this behaviour?

oof. It just took me five minutes to find this bug again. because you appended this behavior to the Deferred variable problem. this is not good I am going to move this to a new one: #278

I will remove the option because it will already work with current versions of Cardbook and there is no risk in hard coding the lowercase on lookup (which I didn't know whether it was safe to assume this, because I saw mixed case for emails in the CardBook UI was possible)

[RealRaven2000]

note to self: never allow piggybacking new bugs in existing issues. It's a horrible rabbit hole. I usually tell my users to open a new issue, this one came back to bite me after I forgot to say it.

[RealRaven2000]

PS: I also already translated the full change log, hopefully I can just insert the link to the new issue without having to repeat the whole process for 20 languages.