Provide Recipe for Implementing "Smart Kerning"

[stevemolitor]

The Commit Mono font has a feature that it calls "smart kerning". Monaspace has a similar feature that it calls "texture healing." Characters are moved around slightly, or swapped out for a slightly larger or smaller character to even out the spacing, for example, when a wide character like an 'm' is followed by a narrow character like an 'l'.

I made a bit of progress getting that to work in Emacs using ligature.el. On the CommitMono website, one of their "smart kerning" examples shows what happens when you type "Moi". I added this:

(ligature-set-ligatures 'emacs-lisp-mode `("Moi"))

Now I see the same effect in Emacs. As you type the "i", the "o" shifts away from the "M" a bit.

I found the Commit Mono kerning feature in its repo here. It classifies common ascii characters into "narrow", "normal", "larger", and "widest" sizes, and repositions them based on what they appear next too.

I could write a script to generate all the combinations for use with ligature.el. It could be a large number of combinations. Then again ligature.el supports regexp ligature specs so maybe it wouldn't be so bad.

However, I noticed that using ligature.el I had to add 3 character ligatures to get it to work: "Moi", not "oi".

I'm a little fuzzy on the order of precedence of both this kearning feature, and when multiple ligatures could match in ligature.el.

Can you think of a recipe to get this sort of feature working with ligature.el? This kerning feature is starting to pop up under different names in new monospace fonts. I can see how some may not like having characters shift around, but OTOH it'd be useful when showing code in presentations if nothing else. It does read very well.

Vscode supports it OOTB; it'd be nice if it could work in Emacs also even if it requires some elbow grease.

[mickeynp]

The problem is, Emacs does not support font features. So ligature.el works around that by asking that you tell it how to ligate things, and then it tells Emacs how to do it. I'm not sure how this new feature actually works: maybe it defaults to on and that's why it works? If you just want it working, you can try

https://github.com/twardoch/fonttools-opentype-feature-freezer

And see if that simplifies things? Not sure if there's a quick solution to this problem as Emacs does not support font features.

[stevemolitor]

Hi @mickeynp thanks for the response.

Yeah, the "smart kerning" feature is ss05 and I had downloaded the font with that feature and the ligature features baked in. I verified with https://github.com/MuTsunTsai/fontfreeze and sure enough, the ss05 feature is on in the downloaded font, and I can see the "smart kerning" effect when I type in the font freeze (and VSCode), but not in Emacs.

The "smart kerning" feature works by seeing if the previous and next character are in different size classes than the current character, and swapping out the current character with one shifted to the right or the left as appropriate.

I tried jumping down a level:

(dolist (char/ligature-re
         `((?o "Com")
           (?m "mmi")))
  (apply (lambda (char ligature-re)
           (set-char-table-range composition-function-table char
                                 `([,ligature-re 1 font-shape-gstring])))
         char/ligature-re))

I used 1 to tell it to look one character prior to the current character when evaluating the regexp (or 3 character string in this case).

The idea is that if this worked, I could write some elisp to loop thru the size class characters and do this more completely.

It half worked. Typing "Com" on one line produces the smart kerning effect: as soon as I type the "m" the "o" shifts away from it to make room. Similarly, if I type "mmi" on a separate line, the second "m" shifts when I type the "i".

However, typing "Commit" does not work. The "Com" bit works, but nothing happens when I type "mmi". Apparently, I've created 3 character, non-overlapping ligatures, and the first "m" is owned by the "Com" and not accounted for in "mmi". Or something like that.

This composition-function-table stuff is pretty opaque to me though. I'm not sure if what I'm trying to do is possible or not.

[stevemolitor]

Here are some screenshots to illustrate the issue. Using the above code, if I type "Com" it works as expected - the "o" shift slightly to the left when I type the "m"

If I type "mmi" on a separate line it also works - the second "m" shifts to the left when I type the "i"

However, if I type "Commit", the "Com" works (the "o" shifts"), but nothing happens when I type "mit". If I use three m's and type "Commmit" I do see the effect on the final "mit":

So again, it seems I've created 3 character, non-overlapping gifs, which is not the effect I want.

For comparison, here it is in VSCode, typing "Commit". Various shifting effects happen as expected:

[khaledhosny]

https://github.com/harfbuzz/harfbuzz/discussions/4490#discussioncomment-7566290

[stevemolitor]

Eli kindly replied to me on emacs-dev. When Emacs sees a regexp in the composition table it treats all characters matched as being "taken care of". Emacs will not look at them again for other rules. It will restart examining from the first character after those which matched.

Instead you need to match against whole "words" of text, which could be very slow.

This will make the "smart kerning" feature work for Commit Mono but will likely have performance issues:

(set-char-table-range
 composition-function-table
 t
 `(["[  ,-.:;A-Z_a-z]+" 0 font-shape-gstring]))

For ligatures you'll need something like this:

(set-char-table-range
 composition-function-table
 t
 `(["[  ,-.:;A-Z_a-z><=!&|+-?/\\]+" 0 font-shape-gstring]))

Removing the space character from the regexp might speed things up a bit, at the cost of not correctly kerning certain character combos that occur at the end of a line (kind of an edge case).

So far I haven't noticed any performance issues on my M1, but I haven't tried with super large files yet either. So, caveat emptor.

[stevemolitor]

I noticed that adding certain characters to the regexp messed up things like dired. Here's the setup I'm currently using for "smart kerning" with Commit Mono:

EDIT 2023-12-19: Here's a slightly simplified version, with no leading space as that messes up corfu's child frames

;; set safe composition table that works in all modes:
(set-char-table-range composition-function-table t `(["[,-.;A-Z_a-z]+" 0 font-shape-gstring]))

;; creates and sets a buffer local composition table to value
(defun set-buffer-local-composition-table (value)
  (let ((table (make-char-table nil)))
    (set-char-table-range table t `([,value 0 font-shape-gstring]))
    (set-char-table-parent table composition-function-table)
    (setq-local composition-function-table table)))

;; sets prog-mode composition table - includes programming ligatures
(defun set-prog-mode-table ()
  (set-buffer-local-composition-table "[-.,:;A-Z_a-z><=!&|+?/\\]+"))

;; Turn on ligatures in all programming modes:
(add-hook 'prog-mode-hook #'set-prog-mode-table)

I noticed that matching against multiple spaces messed up dired and vertico-buffer for some reason. Commit Mono doesn't do any sliding around of spaces obviously, but we do need to include a leading space in the regexp to handle cases like " am", since the smart kerning works on 3 character sequences, comparing the current character with the previous and next characters.

For programming modes, I also include a buffer-local composition table with all the characters for ligatures. If you don't like ligatures you can skip that bit.

Eli advised that regexps that grab more characters are better, so that more characters can be passed to HarfBuzz in one go. So far this config is working well for me. However, be forewarned that it could cause performance issues with redisplay.

I created this config by looking at the Commit Mono source to see which characters it uses for smart kerning. I did quickly try this setup with the Monaspace fonts, which use a similar technique they call "textual healing". It seemed to work as designed but this config isn't really tuned for that font.

I could extend this to support mode-specific ligatures. Ligature.el could be extended to support this sort of configuration by changing the string-to-char call here to also allow t as an index. However, this is not a recommended way to use the composition table so I'm not sure if that's desirable. cc @mickeynp

[mickeynp]

Nice sleuthing, Steve! Thanks for updating me

[arjaz]

The proposed solution unfortunately messes up avy pretty badly for me.

For anyone wanting this feature and experiencing the same avy visual artifacts, here's a good enough solution that I'm using: temporarily disable the composition table around the avy-jump invocation

(defun toggle-safe-composition-table--around (old-fn &rest args)
  "Disable the composition table around a function invocation. Useful to prevent weird avy artifacts."
  (let ((visible-buffers (mapcar #'window-buffer (window-list))))
    (dolist (b visible-buffers)
      (with-current-buffer b
        (unset-safe-composition-table)))
    (let ((res (apply old-fn args)))
      (dolist (b visible-buffers)
        (with-current-buffer b
          (set-safe-composition-table)))
      res)))

(advice-add 'avy-jump
            :around
            #'toggle-safe-composition-table--around)