Auto-correct in the message compose editor

[Betterbird]

People have requested auto-correct to turn -- into –, quotes in to smart quotes, and perhaps a few things more. Apparently there is no bug for this in BMO, https://mzl.la/3LkbWIj, other than enabling auto-correct for Mac (bug 606156). Seems like a big project which would need some funding incentive.

[tdulcet]

@rugk and I do have a Thunderbird add-on on ATN to autocorrect Unicode smart quotes, hyphens over 80 other symbols: https://github.com/rugk/unicodify, but this feature is still experimental, so any "funding incentive" would certainly be helpful to get it across the finish line.

For a while, we have been looking for an expert in contentEditable elements and designMode like the Thunderbird compose window uses to find a solution for this issue, which is our last blocker for enabling the autocorrect feature by default. That link only mentions Firefox, but the issue is also a problem in Thunderbird (see bug 1777183). Maybe the Betterbird project has connections to someone with the requisite knowledge in WYSIWYG editors to finally find a workable solution...

[Betterbird]

Our project leader worked on the Mozilla editor a while ago. But since then, the thing has been completely restructured.

https://discourse.mozilla.org/t/how-to-get-the-caret-position-in-contenteditable-elements-with-respect-to-the-innertext/91068 is tricky. We once upon a time looked how BlueGriffon, the HTML editor was doing it when switching from WYSIWYG to the HTML view, and they insert a special string that they then look for. Due to this issue, the add-on ThunderHTMLedit doesn't position the caret properly in the HTML view.

Just occurred to me: ThunderHTMLedit in an ancient version was using https://searchfox.org/mozilla-central/source/editor/nsIEditActionListener.idl https://searchfox.org/mozilla-esr45/source/editor/txmgr/nsITransactionListener.idl (looks like it's no longer there, perhaps renamed or replaced). Would those help? The former seems to tell you where the last insert/delete was done.

What's the issue in https://bugzilla.mozilla.org/show_bug.cgi?id=1777183 exactly? At a brief glance, your add-on changed the enter-behaviour.

P.S./EDIT: We once upon a time (in 2015) had contact with staff from www.ephox.com. Maybe the contact still works there.

[tdulcet]

Would those help? The former seems to tell you where the last insert/delete was done.

This likely would help if the autocorrect feature was going to be integrated into Thunderbird/Betterbird itself (your request with this issue) or if someone was going to create an Experiment API to support it (like what the ThunderHTMLedit add-on uses). However, as a pure Mail/Web Extension, we are of course limited to just the standard HTML and JS APIs, which make it extremely difficult to implement this autocorrect functionality. Maybe we need to request a new Web Extension API.

What's the issue in https://bugzilla.mozilla.org/show_bug.cgi?id=1777183 exactly? At a brief glance, your add-on changed the enter-behaviour.

No, this was a regression in the Mozilla editor which is of course used by both Firefox and Thunderbird. Unfortunately, the issue is only reproducible with the nonstandard "Body text" mode, which it seems is only used by Thunderbird and so not covered by any of Firefox's automated tests. Based on my use of mozregression, it looks like the issue was introduced in FF/TB 102 with bug 1766355 and later fixed in FF/TB beta 104 with bug 1778091, but the fix was never uplifted to ESR 102 (they marked it as "wontfix"). 😕 See https://github.com/rugk/awesome-emoji-picker/issues/129 for more information. Maybe you should consider including that patch in Betterbird.

The reason I mentioned this bug, is that our current approach of inserting the null character to determine the caret position causes some weird and hard to reproduce behavior. I strongly suspect most if not all of this behavior is due to other unreported bugs in the Mozilla editor, but I have been unable to determine a consistent STR to report them myself on BMO. Inserting and then quickly removing a character on every keystroke must not be something a lot of websites are doing, so it hits a lot of untested edge cases in the Mozilla code. If you want to see what I mean, I would encourage the Betterbird team to install our Unicodify add-on from ATN, enable the "Unicode autocorrect" feature on the options page and then try typing out a few long e-mails. The autocorrect feature should work as expected, but you would likely experience some unwanted behavior. Feedback is welcome.

Anyway, even if all these bugs were fixed, the null character approach is obviously problematic, which is why we have been looking for a better and more robust way to determine the caret position. We actually have two add-ons, this Unicodify add-on for Unicode autocorrection and the Awesome Emoji Picker add-on for emoji autocorrection (bug 1727498), which both share the same code to do the autocorrection. If the user installs both add-ons, that means it has to insert two characters on every keystroke, which is of course also not scalable.

Any help the Betterbird project could provide to find a workable solution to support autocorrection in Thunderbird/Betterbird would be greatly appreciated.

[Betterbird]

Bug 1778091 also fixes a regression caused by bug 1730429. Sadly the M-C editor is in constant flux, or has been for a while, and it's impossible to backport bug 1778091, we tried and failed. Hence in BB we backed out bug 1730429 (check our release notes).

If you want to remain a pure WebExtension, you will have to request an API for your purposes.

Other than that, I'll have to completely disappoint you: we won't add support for any add-ons. Maybe in the fullness of time Mozilla/TB or our project will implement autocorrection.

If you're still interested in how to best find the caret position, you could inspect some open-source HTML editors or I can try to put you into contact with Ephox.

[tdulcet]

If you want to remain a pure WebExtension, you will have to request an API for your purposes.

I just filed Bug 1902858, which is a more general request to support this use case. If they approve it, I could then make a more specific request for the new APIs that are needed. I am hoping I can get TB involved in this process as well...

https://searchfox.org/mozilla-esr45/source/editor/txmgr/nsITransactionListener.idl (looks like it's no longer there, perhaps renamed or replaced).

Looks like this was removed in bug 1767876.