Add support for simple regex transformations in tab stops.

[savetheclocktower]

Note: this is my first deep-dive into adding an Atom feature and I’m happy to get pushback on code formatting, architectural choices, and the like.

Description of the Change

This implements a slimmed-down version of snippet transformations. TextMate’s version of this feature is documented here.

For example, this snippet:

<${1:p}></${1/[ ]+.*$//}>

can be used like this:

Design

I went through a bunch of possible ways to design this. Here’s what I landed on:

• Update the pegjs file so that we can parse the new syntax.
• Introduce the concept of an Insertion, which belongs to a tab stop and can either mirror or transform its input.
• Upon expanding a snippet or moving to a new tab stop, we check if the active tab stop has at least one transformation. If so, we observe the buffer so that every transactional update that happens while that tab stop is active (except for the transactions we initiate) results in a recomputation of the transform output.
• For undo and redo to work the way we want, we have to take the changes that occur as a result of transformations and fold them into the change that kicked off the transformation in the first place. And we need to spy on the history provider so that we can skip trying to reapply transformations upon undo or redo.

Worth noting

Mirrored tab stops have always been treated like multiple selections…

…and that doesn’t change. But transformed tab stops are not treated like selections, because I think a user wouldn’t reasonably expect the same input to produce different output when applied to multiple selections.

Transformed tab stops cannot have placeholders; the first transformation gets applied as soon as the snippet is expanded, so any placeholder value would likely be lost immediately.

What is and isn't included

As explained in this comment, I wanted to go minimal here. The find-and-replace syntax roughly corresponds to the two arguments you'd pass to String#replace (the regex is automatically given the global flag). There's one extra feature that I brought over from TextMate format strings: the ability to control case through the \u, \l, \U, \L, and \E flags:

• \u and \l will force the next character to uppercase or lowercase, respectively.
• \U and \L will force the rest of the string to uppercase or lowercase, respectively (or until we encounter a different case directive or \E).
• \E stops the effects of \U and \L.

There are lots of other features of TextMate format strings, but a lot of them are obscure. I went with this small extension because it's quite useful and is also supported by Sublime Text's snippet transformation syntax. Other features, like conditional insertions, could be added rather easily in the future just by updating the parser and the Insertion class.

Alternate Designs

I don’t like how a snippet expansion needs to listen to TextBuffer changes in order to do transformations, but I didn’t see another way. An asynchronous listener doesn’t act quickly enough. But we do listen at the transaction level (TextBuffer#onDidChangeText) rather than the keystroke level (TextEditor#onDidChange).

If there is a simpler way to do this, I’m all for it.

Benefits

Makes snippets far more useful. There are a couple more examples in the (duplicate) ticket that I opened.

Possible Drawbacks

Not sure. Increased code complexity, perhaps?

This PR doesn't change any existing behavior — barely any of the existing tests even needed to be touched — and the theoretical performance penalty of observing TextBuffer transactions should only be encountered when using a snippet with a transformation; for existing use cases there should be no performance regression.

Applicable Issues

Ticket #40 contains the original request.

[nathansobo]

Published as snippets@1.2.0. Thanks for the great work adding this. You combined a bunch of APIs in a pretty complex piece of code to add a really cool feature. I've gone ahead and given you commit bit on this repository ⚡️.

[savetheclocktower]

Thanks! I'll revisit this when 1.25 is out, and the undo/redo thing will be on my radar as well.

[nobleach]

Forgive my misunderstanding but, is this actually available in non-beta releases? I see it's published in snippets@1.2.0 (current is 1.3.3).

[savetheclocktower]

Yeah, it went out with Atom 1.25.0.

[nobleach]

That's great news! Looks fantastic. I'll dig around in the source a bit to see how to use it. Thanks!