Allow macro invocations in patterns

[kyrime]

Currently, macro invocations in patterns cause parsing errors. This was partially addressed by #24, but the fix was not general enough to handle all cases.

Examples that do not currently parse correctly but are accepted by the compiler:

macro_rules! foo {
    () => {
        _
    };
}

fn main() {
    // Macro in a `let` pattern.
    let foo!() = 3;

    // More than one macro invocation in a `match` branch.
    match () {
        foo!() | foo!() => ()
    }
}

This results in the following output from tree-sitter-cli parse:

Expand

(source_file [0, 0] - [15, 0] (macro_definition [0, 0] - [4, 1] name: (identifier [0, 13] - [0, 16]) (macro_rule [1, 4] - [3, 5] left: (token_tree_pattern [1, 4] - [1, 6]) right: (token_tree [1, 10] - [3, 5]))) (function_item [6, 0] - [14, 1] name: (identifier [6, 3] - [6, 7]) parameters: (parameters [6, 7] - [6, 9]) body: (block [6, 10] - [14, 1] (line_comment [7, 4] - [7, 32]) (let_declaration [8, 4] - [8, 19] pattern: (tuple_struct_pattern [8, 8] - [8, 14] type: (identifier [8, 8] - [8, 11]) (ERROR [8, 11] - [8, 12])) value: (integer_literal [8, 17] - [8, 18])) (line_comment [10, 4] - [10, 58]) (expression_statement [11, 4] - [13, 5] (match_expression [11, 4] - [13, 5] value: (unit_expression [11, 10] - [11, 12]) body: (match_block [11, 13] - [13, 5] (ERROR [12, 8] - [12, 30] (macro_invocation [12, 8] - [12, 14] macro: (identifier [12, 8] - [12, 11]) (token_tree [12, 12] - [12, 14])) (match_arm [12, 17] - [12, 30] pattern: (macro_invocation [12, 17] - [12, 23] macro: (identifier [12, 17] - [12, 20]) (token_tree [12, 21] - [12, 23])) value: (unit_expression [12, 27] - [12, 29]))))))))) src/main.rs 0 ms (ERROR [8, 11] - [8, 12]) Total parses: 1; successful parses: 0; failed parses: 1; success percentage: 0.00%

After this patch, the example parses correctly:

Expand

(source_file [0, 0] - [15, 0] (macro_definition [0, 0] - [4, 1] name: (identifier [0, 13] - [0, 16]) (macro_rule [1, 4] - [3, 5] left: (token_tree_pattern [1, 4] - [1, 6]) right: (token_tree [1, 10] - [3, 5]))) (function_item [6, 0] - [14, 1] name: (identifier [6, 3] - [6, 7]) parameters: (parameters [6, 7] - [6, 9]) body: (block [6, 10] - [14, 1] (line_comment [7, 4] - [7, 32]) (let_declaration [8, 4] - [8, 19] pattern: (macro_invocation [8, 8] - [8, 14] macro: (identifier [8, 8] - [8, 11]) (token_tree [8, 12] - [8, 14])) value: (integer_literal [8, 17] - [8, 18])) (line_comment [10, 4] - [10, 58]) (expression_statement [11, 4] - [13, 5] (match_expression [11, 4] - [13, 5] value: (unit_expression [11, 10] - [11, 12]) body: (match_block [11, 13] - [13, 5] (match_arm [12, 8] - [12, 30] pattern: (match_pattern [12, 8] - [12, 23] (or_pattern [12, 8] - [12, 23] (macro_invocation [12, 8] - [12, 14] macro: (identifier [12, 8] - [12, 11]) (token_tree [12, 12] - [12, 14])) (macro_invocation [12, 17] - [12, 23] macro: (identifier [12, 17] - [12, 20]) (token_tree [12, 21] - [12, 23])))) value: (unit_expression [12, 27] - [12, 29])))))))) Total parses: 1; successful parses: 1; failed parses: 0; success percentage: 100.00%

The motivating example can be seen at https://github.com/helix-editor/helix/blob/e04bb8b8915bfe1df1a5ee7a8750f2589f2aae06/helix-term/src/ui/picker.rs#L502-L514 where there are multiple macro invocations per match branch, leading to incorrect syntax highlighting in an editor using tree-sitter-rust.

[maxbrunsfeld]

Thanks for the fix!