Expand cursorless talon command api

[pokey]

We'd like for users to be able to define more complex custom grammars. To do so, they'll need full access to the types of commands supported by cursorless, whereas today they can just do a single target, with no extra arguments

• [ ] Support multiple targets
• [ ] Support extraArgs
• [ ] Support getText action
• [ ] Document target api representation and allow users to pass in home-made targets
• [ ] Require user to specify api version number if they use home-made targets
• [ ] https://github.com/cursorless-dev/cursorless/issues/2246
• [ ] Add actions.user.cursorless_implicit_target_command?
• [ ] Important to include version number for backwards compatibility

Proposal

See https://github.com/cursorless-dev/cursorless/issues/492#issuecomment-1810885797 below

Old proposal

• [ ] Create a new action called cursorless_target that accepts a list of scope ids, followed by an optional target dict. Can accept None as last arg as well and that's equivalent to "this" / omitting mark. If it gets a target it prepends the containing scope modifiers to the target's modifiers
• [ ] In the future we can support things like cursorless_target("every", "collectionItem", "this")

Example

complete [<user.cursorless_target>]:
    target = user.cursorless_target("collectionItem", cursorless_target)
    user.cursorless_command("setSelectionBefore", target)
    key(right:3 delete x)

More verbose alternative

complete [<user.cursorless_target>]:
    user_target = cursorless_target or "this"
    item_modifier = user.cursorless_containing_scope("collectionItem")
    target = user.cursorless_add_modifier(item_modifier, user_target)
    # Could reduce the above two lines to:
    # target = user.cursorless_expand_to_scope("collectionItem", user_target)

Use cases

Checking a box in a markdown checklist

See also #452 and #453

Consider if we want to support extending/overriding other internal cursorless features.

eg support for when focus is not on the text editor. https://github.com/cursorless-dev/cursorless/pull/1717#discussion_r1284083455

[tararoys]

Allow custom grammars for reformat action is what I vote!

[josharian]

Sample use case:

hug <phrase>$:
    mimic("chuck leading " + phrase)
kiss <phrase>$:
    mimic("chuck trailing " + phrase)
(cosy|cozy) <phrase>$:
    m = "chuck leading " + phrase
    m = m + " and trailing " 
    m = m + phrase
    mimic(m)

[josharian]

Since it often helps to have concrete, real world use cases written down, here is a running list of things I am currently implementing using mimic. (Will update it over time.)

pre line <x>
chuck leading <x>
chuck trailing <x>

[josharian]

GitHub failed to backlink this comment, so here it is manually: https://github.com/cursorless-dev/cursorless/discussions/1655#discussioncomment-6570805

[pokey]

update from meet-up: there are a couple approaches

Use csvs

We could have custom modifier and action csvs which map from a spoken form to some representation of compound modifiers / actions. Here are a couple possibilities for representing "chuck previous char" as a custom action:

• remove previous character
• remove relative(-1; character)
• remove relative(-1) character
• remove <~ relative(-1; character)
• relative(-1; character) | remove (inspired by jq)

Benefit of using identifiers rather than spoken form is that they can change spoken form without breaking these, at the expense of being a bit harder to read / write

In a csv this would be something like

experimental/custom_actions.csv:

Spoken form, value
shave, remove <~ relative(-1; character)

For custom modifiers, we could have another csv where it's the same syntax but without the leading action. Eg

experimental/custom_modifiers.csv:

Spoken form, value
duet, nScopes(2; token)

We could use this syntax in other places, eg

• To pass in to an action eg user.cursorless_command("remove previous character", cursorless_target)
• For use in some kind of "cursorless scripting language" (issue to follow, but in the meantime see sam for inspiration)

Could have a command to automatically generate this syntax, eg "dump modifier two tokens" to type out nScopes(2; token), or "dump action chuck previous char"

The benefit of this csv approach is that it's compact, and these are super easy to share / find in other people's repos, so even if people struggle a bit to read / write the syntax, power users could come up with cool compound modifiers / actions that others steal

Use a set of Talon actions

Another approach is to expose actions that users can use to build up targets. Eg

shave <user.cursorless_target>:
    target = user.cursorless_modifier_relative("character", -1, cursorless_target)
    user.cursorless_action_remove(target)

Note that it's not immediately obvious how to use this approach for custom modifiers, eg "duet" above.

The benefit of this approach is that we don't need to have a custom parser, and the user can easily use it in Python, etc. The syntax will also be familiar for users of Talon / Python

One disadvantage is that we potentially have less flexibility to change things, because we're locked into this api, whereas with our custom grammar we have more wiggle room.

We could add version numbers, eg user.cursorless_v6_modifier_relative, but then we run into an issue where a user might take something returned from user.cursorless_v6_modifier_relative and try to pass it to user.cursorless_v7_action_remove. We effectively have an api surface where each function is kind of its own "api surface island" that needs to be able to interact with anything else. Which we could solve probably with a lot of version numbers on every little object, but that seems painful to maintain

[josharian]

Using the spoken form directly is pretty appealing though. What if it always parsed used the default spoken forms? It’s not too hard to look up the default spoken form, once you understand cursorless well enough to have your own dialect. And they’d be fully transferable from one user to another.

And if needed you could use sql-like placeholders to indicate substitution locations (lots of options here).

[pokey]

Interesting idea to support default spoken form, though that locks us in to a set of default spoken forms, and they also might be surprised when they can't use their own. What if we just supported the exact same grammar as spoken, but using canonical ids, and supported a way to automatically generate from spoken form, as described above?

The annoyance here is we'd need to implement our own command parser, rather than relying on Talon's as we do today, and there are likely things they may try to do that wouldn't quite work because they're stopping in the middle of some construct. Eg they might try to map "dock" to "chuck tail" and expect to say "dock funk", but really they've stopped in the middle of a modifier

[josharian]

Lots of dimensions on which to evaluate these:

• Stability: custom jq-like grammar > canonical ids > default spoken form > python api > custom spoken form
• Readability: custom spoken form > default spoken form > python api / custom jq-like grammar / canonical ids
• Transferability from one user to another: default spoken form / python api / custom jq-like grammar / canonical ids > custom spoken form
• Implementation difficulty: ?

there are likely things they may try to do that wouldn't quite work because they're stopping in the middle of some construct

yeah. good error messages from the parser could help a lot here, though.

[josharian]

Use case from #1514:

"trade <modifier> <mark>" -> "swap <modifier> <mark> with its next <modifier>"

(Although I may have butchered the grammar, which would itself be a useful data point.)

Given f(a, b), 'trade arg air" generates f(b, a).

[pokey]

Lots of dimensions on which to evaluate these:

• Stability: custom jq-like grammar > canonical ids > default spoken form > python api > custom spoken form
• Readability: custom spoken form > default spoken form > python api / custom jq-like grammar / canonical ids
• Transferability from one user to another: default spoken form / python api / custom jq-like grammar / canonical ids > custom spoken form
• Implementation difficulty: ?

I like this way of thinking about the problem

[pokey]

Example from slack

1. "take next instance"
2. "copy block"

Wanted to be able to do something like:

copy next:
   user.cursorless_action("take next instance")
   user.cursorless_action("copy block")

[pokey]

Api

Custom actions

Add a new type of action:

interface ParsedActionDescriptor {
  name: "parsed";
  content: string;
  targets?: PartialTargetDescriptor[];
}

Here is how we would use it in a Talon file:

kill: user.cursorless_custom_command("chuck block")
kill <user.cursorless_target>: user.cursorless_custom_command("chuck block <target>", cursorless_target)
my before <user.cursorless_target>: user.cursorless_custom_command("bring this before <target>", cursorless_target)
my before <user.cursorless_target> and <user.cursorless_target>: user.cursorless_custom_command("bring <target1> before <target2>", cursorless_target_1, cursorless_target_2)

Here is the command payloads that would come from the above rules, respectively:

{
  action: {
    name: "parsed",
    content: "chuck block",
  }
}

{
  action: {
    name: "parsed",
    content: "chuck block <target>",
    targets: [{...}],
  }
}

{
  action: {
    name: "parsed",
    content: "bring this before <target>",
    targets: [{...}],
  }
}

{
  action: {
    name: "parsed",
    content: "bring <target1> before <target2>",
    targets: [{...}, {...}],
  }
}

Might also want to have csv version for actions, but let's leave that out for now

kill, chuck block

Custom modifiers

Add a new type of modifier:

interface ParsedModifier {
  type: "parsed";
  content: string;
}

Then add a new csv file talon-side called custom_modifiers.csv:

duet, two tokens

Then user could use "duet" as a modifer. The modifier would correspond to:

{
    type: "parsed",
    content: "two tokens",
}

The user could say eg "take duet air"

Implementation

• In addition to the new modifier and action type above, add a new type of mark called called PlaceholderMark (see also https://github.com/cursorless-dev/cursorless/issues/2096). It just has { type: "placeholder" }

• Flesh out grammar in https://github.com/cursorless-dev/cursorless/blob/main/packages/cursorless-engine/src/customCommandGrammar/grammar.ne

  Something like:

  @preprocessor typescript
  @{%
  import { capture } from "../../util/grammarHelpers";
  import { lexer } from "../lexer";
  %}
  @lexer lexer
  
  main -> command
  
  command -> %simpleAction target
  
  target -> modifer:* mark
  mark -> %placeholderTarget {%
  () => ({ type: "placeholder" })
  %}
  
  # --------------------------- Scope types ---------------------------
  scopeType -> %simpleScopeTypeType {% capture("type") %}
  scopeType -> %pairedDelimiter {%
  ([delimiter]) => ({ type: "surroundingPair", delimiter })
  %}

• When running a command, if the action is type "parsed", we parse content, and the output may have placeholder marks. For each placeholder mark we find, we replace the placeholder mark with a TargetMark containing the corresponding Target in the targets list passed to the "parsed" target. Note that a TargetMark runs a full pipeline on an arbitrary target (including compound targets), and then the output of that flows through the rest of the pipeline. This would make it so that if the user defined "kill" as "chuck block" above, they could say "kill air and bat past cap" and it would just work (ie they would all be treated as blocks). Note that inference wouldn't work quite the same as normal inference, because if they said "kill air and token bat", it would still convert "token bat" to a block, unlike "chuck block air and token bat". But that seems like a reasonable trade-off for a nice simple implementation

[brollin]

Excited for this to be implemented! My use case is very similar to aforementioned ones. I would like to be able to write a quick+dirty talon command that would accomplish:

"post next block this"

for purposes of navigating a markdown file. Very random use case but along the lines of the markdown "complete" to check a markdown checkbox.