tree-sitter / tree-sitter-julia

Julia grammar for Tree-sitter
MIT License
93 stars 32 forks source link

Error when parsing escape sequence in non-standard String literals #89

Closed Klafyvel closed 1 year ago

Klafyvel commented 1 year ago

Hello, and thank you for providing this grammar.

There seems to be an issue with tree-sitter-julia when parsing non-standard String literals that contain escape sequences.

Example code that causes the parser to fail:

function force_word_context(parents, tokens, i)
    k = kind(first(parents))
    if K"InlineCode" ∈ parents
        kind(tokens[i]) ∉ [K"`", K"\\"]
    elseif k == K"Verbatim"
        kind(tokens[i]) != K"@"
    else
        false
    end
end
If that can help, here is the output of the TreeSitterPlayground plugin in nvim. ``` function_definition [0, 0] - [9, 3] name: identifier [0, 9] - [0, 27] parameters: parameter_list [0, 27] - [0, 47] identifier [0, 28] - [0, 35] identifier [0, 37] - [0, 43] identifier [0, 45] - [0, 46] assignment [1, 4] - [1, 28] identifier [1, 4] - [1, 5] operator [1, 6] - [1, 7] call_expression [1, 8] - [1, 28] identifier [1, 8] - [1, 12] argument_list [1, 12] - [1, 28] call_expression [1, 13] - [1, 27] identifier [1, 13] - [1, 18] argument_list [1, 18] - [1, 27] identifier [1, 19] - [1, 26] if_statement [2, 4] - [8, 7] condition: binary_expression [2, 7] - [2, 32] prefixed_string_literal [2, 7] - [2, 20] prefix: identifier [2, 7] - [2, 8] operator [2, 21] - [2, 24] identifier [2, 25] - [2, 32] call_expression [3, 8] - [3, 23] identifier [3, 8] - [3, 12] argument_list [3, 12] - [3, 23] index_expression [3, 13] - [3, 22] identifier [3, 13] - [3, 19] vector_expression [3, 19] - [3, 22] identifier [3, 20] - [3, 21] ERROR [3, 24] - [5, 31] bare_tuple [3, 29] - [4, 26] prefixed_string_literal [3, 29] - [3, 33] prefix: identifier [3, 29] - [3, 30] prefixed_string_literal [3, 35] - [4, 26] prefix: identifier [3, 35] - [3, 36] escape_sequence [3, 38] - [3, 40] suffix: identifier [4, 18] - [4, 26] alternative: else_clause [6, 4] - [8, 0] boolean_literal [7, 8] - [7, 13] ```
savq commented 1 year ago

If "\\" parses correctly but K"\\" doesn't, then it might be a problem with the scanner.

The scanner handles strings and non-standard strings a bit differently because the latter doesn't allow interpolations. There might a small logic error causing the scanner to not break when encountering the escape sequence. This shouldn't be hard to fix.

Thanks for the report!