Switch to a token stream rendering model

[mkremins]

Rather than attempting to represent nested Clojure forms directly in the DOM as nested HTML elements, one possible alternative is to flatten each potentially nested form into a stream of tokens prior to rendering. Tokens would be split across explicit DOM elements corresponding roughly to lines of text in a traditional editor; within each line, tokens would be rendered flatly from left to right in the manner of plain text.

This approach is interesting primarily because it seems to solve many of the existing layout problems, which are mostly brought on by our ongoing attempt to make nested DOM elements look like plaintext with explicit whitespace layout. In particular, this approach should make it much easier to flatten sequences of closing collection delimiters onto a single line. It may also simplify the implementation of extensible rendering and user-defined layout models for specific classes of forms.

Potential difficulties mostly involve our current assumption that the tree structure of Clojure forms is roughly mirrored by the tree structure of the DOM. CSS rules targeting semantically invalid forms like valueless map keys and nested anonymous functions will no longer apply, since the tree structure of the backing forms will be absent from the emitted HTML. Structural integrity checks will need to move from CSS into ClojureScript if they are to continue working.

[mkremins]

Commits 6de25159076461d72598dcec3d354168525468b9 through d079095cade06bd8343c9e6b98b33b8a1c409d41 comprise a satisfactory implementation of token-stream based rendering.