evincarofautumn / kitten

A statically typed concatenative systems programming language.
http://kittenlang.org/
Other
1.08k stars 39 forks source link

Syntax highlighting mode #189

Open evincarofautumn opened 7 years ago

evincarofautumn commented 7 years ago

We need a mode that accepts a source file and just performs syntax highlighting. Based on #177 it should recover from lexical errors as much as possible so that all inputs produce some reasonable output. Take this fragment as an example:

define hello (

It should be able to produce raw syntax highlighting information consisting of source spans and lexical categories for each token in JSON:

[
{"token": "define", "category": "keyword", "line": 1, "from": 1, "to": 6},
{"token": "hello", "category": "identifier", "line": 1, "from": 8, "to": 12},
{"token": "(", "category": "symbol", "line": 1, "from": 14, "to": 14}
]

It should also be able to directly generate a syntax-highlighted source file as HTML, preserving formatting:

<span class='kitten keyword'>define</span> <span class='kitten identifier'>hello</span> <span class='kitten symbol'>(</span>

Other output formats such as sourcemaps, RTF, and LaTeX would be nice to have but aren’t critical.