Rework command object to support more complex commands and cursorless-style functionality

[david-tejada]

In order to create more complex commands and support cursorless-style functionality I need to rework the command object and more specifically the targets array. Thanks to cursorless most of the thinking has been already done and I can just try to mimic the way they handle that.

There is a fundamental difference to cursorless, and that is the variety of marks. While cursorless only handles with what they call decoratedSymbol, I have at least three different marks that I have to consider:

• Hints: These are one or two letters marks that sit top left of clickable elements.
• Tab marks: They are basically identical to hints but refer to a tab.
• Decorated symbols: Same as in cursorless.

One issue here is that when issuing a command we might not know if we are dealing with a hint or a decorated symbol (with default color). In reality I think commands that deal with the dom should be able to accept any of the two types of marks. Here are some examples with (h) denoting a hint and (s) denoting at decorated symbol:

• copy fine(h) and blue crunch(s): Copy the text content of the element with the hint "f" and the token with the blue mark over the "k".
• copy fine(s) past crunch(h): Copy the text content between the token "f" and the element "k".
• copy fine(s) and crunch(h): Copy the text content of the elements "f" and "k".

With this in mind I think I should have two different types of mark types:

• hint: To be used for tab and maybe direct clicking commands.
• generic: To be used for other commands that deal with the dom. Can be either a hint or a decorated symbol. Can have a property symbolColor.

Here are some hypothetical commands:

Tab manipulation

{
   "spokenForm": "move tab fine past sit before trap",
   "action": {
      "name": "moveTabs",
      "args": []
   },
   "targets": [
      {
         "type": "range",
         "anchor": {
            "type": "primitive",
            "mark": {
               "type": "hint",
               "characters": "f"
            }
         },
         "active": {
            "type": "primitive",
            "mark": {
               "type": "hint",
               "characters": "i"
            }
         },
         "excludeAnchor": false,
         "excludeActive": false
      },
      {
         "type": "primitive",
         "mark": {
            "type": "hint",
            "characters": "t"
         },
         "modifiers": [
            {
               "type": "position",
               "position": "before"
            }
         ]
      }
   ]
}

{
   "spokenForm": "move tab gust and pit and crunch after air drum",
   "action": {
      "name": "moveTabs",
      "args": []
   },
   "targets": [
      {
         "type": "list",
         "elements": [
            {
               "type": "primitive",
               "mark": {
                  "type": "hint",
                  "characters": "g"
               }
            },
            {
               "type": "primitive",
               "mark": {
                  "type": "hint",
                  "characters": "p"
               }
            },
            {
               "type": "primitive",
               "mark": {
                  "type": "hint",
                  "characters": "k"
               }
            }
         ]
      },
      {
         "type": "primitive",
         "mark": {
            "type": "hint",
            "characters": "ad"
         },
         "modifiers": [
            {
               "type": "position",
               "position": "after"
            }
         ]
      }
   ]
}

{
   "spokenForm": "move tab fine past sit before trap",
   "action": {
      "name": "moveTabs",
      "args": []
   },
   "targets": [
      {
         "type": "range",
         "anchor": {
            "type": "primitive",
            "mark": {
               "type": "hint",
               "characters": "f"
            }
         },
         "active": {
            "type": "primitive",
            "mark": {
               "type": "hint",
               "characters": "i"
            }
         },
         "excludeAnchor": false,
         "excludeActive": false
      },
      {
         "type": "primitive",
         "mark": {
            "type": "hint",
            "characters": "t"
         },
         "modifiers": [
            {
               "type": "position",
               "position": "before"
            }
         ]
      }
   ]
}

DOM commands

{
   "spokenForm": "take fine past sit trap",
   "action": {
      "name": "setSelection",
      "args": []
   },
   "targets": [
      {
         "type": "range",
         "anchor": {
            "type": "primitive",
            "mark": {
               "type": "generic",
               "characters": "f"
            }
         },
         "active": {
            "type": "primitive",
            "mark": {
               "type": "generic",
               "characters": "it"
            }
         },
         "excludeAnchor": false,
         "excludeActive": false
      }
   ]
}

{
   "spokenForm": "copy fine past sit",
   "action": {
      "name": "copyToClipboard",
      "args": []
   },
   "targets": [
      {
         "type": "range",
         "anchor": {
            "type": "primitive",
            "mark": {
               "type": "generic",
               "characters": "f"
            }
         },
         "active": {
            "type": "primitive",
            "mark": {
               "type": "generic",
               "characters": "i"
            }
         },
         "excludeAnchor": false,
         "excludeActive": false
      }
   ]
}

{
   "spokenForm": "copy blue fine past red fox sit",
   "action": {
      "name": "copyToClipboard",
      "args": []
   },
   "targets": [
      {
         "type": "range",
         "anchor": {
            "type": "primitive",
            "mark": {
               "type": "generic",
               "symbolColor": "blue",
               "characters": "f"
            }
         },
         "active": {
            "type": "primitive",
            "mark": {
               "type": "generic",
               "symbolColor": "red-fox",
               "characters": "i"
            }
         },
         "excludeAnchor": false,
         "excludeActive": false
      }
   ]
}

@pokey and @AndreasArvidsson I would like to know what you think.

[pokey]

I'm definitely supportive of the direction! Might make sense to drop into one of the Cursorless meet-ups to discuss. Some initial thoughts:

• I'm not sure I understand the distinction between generic and hint
• Not sure it matters, but Cursorless has many types of marks, not just decoratedSymbol. See the "special marks" section of the cheatsheet
• We're planning to change how we handle "destinations", treating them as a first-class object rather than just a modifier on a regular target. See https://github.com/cursorless-dev/cursorless/issues/803. You may want to follow the new model

[david-tejada]

I think the names generic and hint are not the right ones but I think there needs to be a distinction at least on the talon side. On the extension side I have called the tab hints "tab markers"[^1] more than anything to avoid confusion. I think it makes sense to have these two on the talon side too: rango_marker and rango_tab_marker

The reason to not have all just be rango_marker is because something like tab blue air needs NOT to be a valid command. So the capture for rango_marker would mimic cursorless_decorated_symbol. Here rango_characters symbolizes a one or two letter pair, what so far I have called rango_hint but I think rango_characters is more explicit:

@mod.capture(
    rule="[{user.rango_hat_color}] [{user.rango_hat_shape}] (<user.rango_grapheme> | <user.rango_characters>"
)
def rango_marker(m) -> dict[str, Any]:
    """A rango marker"""
    hat_color = getattr(m, "rango_hat_color", "default")
    try:
        hat_style_name = f"{hat_color}-{m.rango_hat_shape}"
    except AttributeError:
        hat_style_name = hat_color
    return {
        "type": "marker",
        "symbolColor": hat_style_name,
        "characters": m.rango_grapheme,
    }

The capture for rango_tab_marker could be something like this:

@mod.capture(rule=("[tab] <user.rango_characters>"))
def rango_tab_marker(m) -> list[dict]:
  return {
        "type": "tabMarker",
        "characters": m.rango_characters,
    }

Not sure it matters, but Cursorless has many types of marks, not just decoratedSymbol. See the "special marks" section of the cheatsheet

I meant visible marks, those are implicit.

I'll be sure to drop by one of the meetings. I have to finish v0.5.0 and I think I want to have some initial cursorless support on v0.6.0. I wanted to write down some initial idea of how the implementation of more complex commands could go. After implementing #101 in its basic form (just to activate tabs) I realized that in order to create more complex commands I would need to work on this.

[^1]: meaning - Difference between "mark" and "marker" - English Language & Usage Stack Exchange