Semantic parsing with fish_indent --dump-parse-tree

[bmalehorn]

echo "hello $argv"

fish_indent --dump-parse-tree sample

 0 -  0  symbol_job_list <3 children>  [0, 19]
 1 -  1    symbol_job_decorator  [0, no src]
 2 -  2    symbol_job_conjunction <2 children>  [0, 18]
 3 -  4      symbol_job <5 children>  [0, 18]
 4 -  6        symbol_optional_time  [0, no src]
 5 -  7        symbol_variable_assignments  [0, no src]
 6 -  8        symbol_statement <1 children>  [0, 18]
 7 - 11          symbol_decorated_statement <1 children>  [0, 18]
 8 - 12            symbol_plain_statement <2 children>  [0, 18]
 9 - 13              parse_token_type_string: "echo"
10 - 14            symbol_arguments_or_redirections_list <2 children>  [5, 13]
11 - 15              symbol_argument <1 children>  [5, 13]
12 - 17                parse_token_type_string: ""hello $argv""
13 - 16            symbol_arguments_or_redirections_list  [18, no src]
14 -  9        symbol_job_continuation  [18, no src]
15 - 10        symbol_optional_background  [18, no src]
16 -  5      symbol_job_conjunction_continuation  [18, no src]
17 -  3  symbol_job_list <2 children>  [18, 1]
18 - 18    parse_token_type_end  [18, 1]
19 - 19  symbol_job_list  [19, no src]

{off    0, len   19, indent  0, kw unknown_keyword, symbol_job_list} [ |echo "hello $argv"
| ]
{off    0, len    0, indent  0, kw unknown_keyword, symbol_job_decorator} [ ||e]
{off    0, len   18, indent  0, kw unknown_keyword, symbol_job_conjunction} [ |echo "hello $argv"|\cJ]
{off    0, len   18, indent  0, kw unknown_keyword, symbol_job} [ |echo "hello $argv"|\cJ]
{off    0, len    0, indent  0, kw unknown_keyword, symbol_optional_time} [ ||e]
{off    0, len    0, indent  0, kw unknown_keyword, symbol_variable_assignments} [ ||e]
{off    0, len   18, indent  0, kw unknown_keyword, symbol_statement} [ |echo "hello $argv"|\cJ]
{off    0, len   18, indent  0, kw unknown_keyword, symbol_decorated_statement} [ |echo "hello $argv"|\cJ]
{off    0, len   18, indent  0, kw unknown_keyword, symbol_plain_statement} [ |echo "hello $argv"|\cJ]
{off    0, len    4, indent  0, kw unknown_keyword, parse_token_type_string} [ |echo| ]
{off    5, len   13, indent  0, kw unknown_keyword, symbol_arguments_or_redirections_list} [ |"hello $argv"|\cJ]
{off    5, len   13, indent  0, kw unknown_keyword, symbol_argument} [ |"hello $argv"|\cJ]
{off    5, len   13, indent  0, kw unknown_keyword, parse_token_type_string} [ |"hello $argv"|\cJ]
{off   18, len    0, indent  0, kw unknown_keyword, symbol_arguments_or_redirections_list} ["||\cJ]
{off   18, len    0, indent  0, kw unknown_keyword, symbol_job_continuation} ["||\cJ]
{off   18, len    0, indent  0, kw unknown_keyword, symbol_optional_background} ["||\cJ]
{off   18, len    0, indent  0, kw unknown_keyword, symbol_job_conjunction_continuation} ["||\cJ]
{off   18, len    1, indent  0, kw unknown_keyword, symbol_job_list} ["|
| ]
{off   18, len    1, indent  0, kw unknown_keyword, parse_token_type_end} ["|
| ]
{off   19, len    0, indent  0, kw unknown_keyword, symbol_job_list} [\cJ|| ]
echo "hello $argv"

Or, use fish_indent --dump-parse-tree, which includes offsets.

• [x] figure out how to do semantic highlighting in VSCode https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide
• [x] run fish_indent --dump-parse-tree on save (or automatically with temp file?)
• [x] parse output
• [x] highlight text, overwriting default highlighting

[bmalehorn]

Got this working, in this branch: https://github.com/bmalehorn/vscode-fish/tree/semantic-tokens

However, fish_indent --dump-parse-tree parsing is very basic and only intended for indenting. For instance, it doesn't distinguish between a regular argument foo and a string "foo". So it can't be used to do all highlighting.