Prototype spatial navigation

Spatial Navigation allows you to navigate by focusing up/down/left/right. It's like mouse keys, but it snaps to the next closest UI element:

Here's a demo that doesn't snap to elements yet. The "cursor" is being controlled by h/j/k/l

spatial_nav_prototype

Some ideas for making this work in roam, and actually emulating vim more closely:

h/j/k/l in "normal mode" focuses the next closest block in that direction
i enters "insert mode", allowing you to edit the block
ctrl-e/y/d/u to scrolls the currently focused panel

I think this is potentially way more intuitive that hint based navigation, while still taking relatively few keystrokes.

Challenges of implementing Spatial Navigation:

UI elements may not line up in a grid

This may make certain UI elements inaccessible if you're strictly navigating only in the cardinal directions, creating "sliding ice block puzzles":

ice_block_puzzle

One way to mitigate this is to focus elements in a cone expanding in the desired direction. That way, an element doesn't have to be perfectly within "line of sight" to be focused.

I don't think this is an issue for for blocks, but it may be an issue for the small buttons sprinkled across Roam's UI.

Moving from a large element to smaller elements is ambiguous

If you move from a large element to two smaller ones, it may be ambiguous which one you want to focus:

+---------------+
|     focus     |
+---------------+
        |
        v

+-----+    +----+
+-----+    +----+

It maybe be acceptable to just focus whatever is on the top/left. A more sophisticated strategy is to focus all viable UI elements as you navigate, and disambiguate by typing a hint.

The hints can be predictable based on their spatial location, like 1-9 starting from the top/left => bottom/right, or mapped to a numpad layout. This eases the cognitive burden of reacting to the hint (see #4 for details), and allows for practice/training.

Selecting small elements inside blocks

Navigation may be unpredictable for UIs with variable density. Your scroll speed would suddenly decrease/increase when you hit a pocket of densely packed UI elements.

For Roam, the example would be if a block contains many small links.

One solution would be to navigate only at the block level, but show nested hints for each link as you switch focus between blocks. This is what the blue outlines in the prototype represent.

How to actually implement the spatial navigation algorithm

elementFromPoint enables getting the element at a certain coordinate. The "mouse keys" esque prototype uses this.

A simple approach would be to use mouse keys under the hood, but mouse key in a certain direction repeatedly until the focused element changes. This may not perform well however.

To implement the "cone" solution for the "ice block puzzle" problem, you could zig-zag in a cone outwards until the focus changes. If the zig-zag hits a corner, abort and stay focused on the current element.

Another algorithm would be to index the block coordinates, and calculate "what block is to the left of me?"

tntmarket / vimmyroam

Prototype spatial navigation #5

Challenges of implementing Spatial Navigation: