Multi-character repetition separators not parseable

[durka]

[Edit: problem has been identified, see next comment]

---

The macro below results in ParseError(None) from parse_str::<MacroRules>. Not very helpful. Is there any way to get more output?

macro_rules! unborrow {
    // =========================================================================================================
    // PRIVATE RULES

    // This rule fires when we have parsed all the arguments.
    // It just falls through to output stage.
    // (FIXME could fold the output rule into this one to reduce recursion)
    (@parse () -> ($names:tt $lets:tt) $($thru:tt)*) => {
        unborrow!(@out $names $lets $($thru)*)
    };

    // Parse an argument and continue parsing
    // This is the key rule, assigning a name for the argument and generating the let statement.
    (@parse ($arg:expr, $($rest:tt)*) -> ([$($names:ident),*] [$($lets:stmt);*]) $($thru:tt)*) => {
        unborrow!(@parse ($($rest)*) -> ([$($names,)* arg] [$($lets;)* let arg = $arg]) $($thru)*)
        //                                            ^                    ^
        // Right here an ident is created out of thin air using hygiene.
        // Every time the macro recurses, we get a new syntax context, so "arg" is actually a new identifier!
    };

    // Output stage for free functions.
    // Assembles the let statements and variable names into a block which computes the arguments,
    // calls the method, and returns its result.
    (@out [$($names:ident),*] [$($lets:stmt);*] ($($meth:ident)::+) $arg1:expr) => {{
        $($lets;)*
        $($meth)::+($arg1, $($names),*)
    }};

    // Output stage for object methods.
    (@out [$($names:ident),*] [$($lets:stmt);*] $($obj:ident).+) => {{
        $($lets;)*
        $($obj).+($($names),*)
    }};

    // =========================================================================================================
    // PUBLIC RULES

    // Macro entry point for object methods.
    ($($obj:ident).+ ($($args:expr),*)) => {
        unborrow!(@parse ($($args,)*) -> ([] []) $($obj).+)
        //                |               |  |   ^ info about the method call, saved for later
        //                |               |  ^ generated let statements
        //                |               ^ generated argument names
        //                ^ arguments to be parsed
    };

    // Macro entry point for free functions.
    ($($meth:ident)::+ ($arg1:expr, $($args:expr),*)) => {
        unborrow!(@parse ($($args,)*) -> ([] []) ($($meth)::+) $arg1)
    };
}

[lukaslueg]

Minimal example is

macro_rules! a {
    ($($m:ident)::+) => {};

}

The problem is the :: on the repetition. : works fine, foo is fine too, :: is flat out.

[durka]

Likely related to #4 then?

[lukaslueg]

In a general sense, yes, yet I believe what we see here are the various intricacies of parsing rust's macro syntax. One will have to investigate.

[durka]

My hunch is you ought to be allowing for a Separator to be simply a token, rather than a Punct (which can only be a single character). But I don't see a Token type in proc_macro2...

[dtolnay]

There isn't a Token type in proc-macro2 or Syn because that is exclusively a macro_rules concept and neither crate is geared around manipulating macro_rules. There is no such thing as multi-character punctuation tokens like :: in the procedural macro token API.

It should be possible to define a parser for multi-character punctuation in this crate, as you have with all the other macro_rules parsing logic.