proc macro API: Making `enum Delimiter` non-exhaustive

rust-lang / rust

Empowering everyone to build reliable and efficient software.

https://www.rust-lang.org

Other

96.73k stars 12.5k forks source link

proc macro API: Making `enum Delimiter` non-exhaustive #54248

Closed petrochenkov closed 5 years ago

petrochenkov commented 6 years ago

... with an eye on extending it with angle brackets < TOKEN_TREES > in the future.

Yes, </> may be unbalanced, but I suspect that we can always consider balanced angle brackets as a group with minimal breakage (based on estimates from https://github.com/rust-lang/rust/pull/53578).

Lang team is seriously intending to remove turbofish from value paths (value::<Type> => value<Type>) despite all the grammar implications, and angle-bracketed groups may be the only way to do it without requiring backtracing from parsers.

Is it realistic to make this change now, when 1.29 with stabilized proc macro API was just released? This stabilization was never announced in release notes or 1.29 blog post, so it may be possible that only infrastructural crates like proc-macro2/syn/... are affected. What are the alternatives if we want to make < ... > a group? cc @dtolnay @alexcrichton @varkor

alexcrichton commented 5 years ago

Changing the stable API now that it's shipped is a non-starter, and unfortunately I don't know how we could retroactively add this.

petrochenkov commented 5 years ago

I guess, as a last resort we could

Keep the true delimiter internally in Group
Report Delimiter::None in fn delimiter if the true delimiter is angle brackets.
Add Group::delim_is_angle_bracket, or enum Delimiter2 + Group::delimiter2.
Paper all this over in proc-macro2/syn and friends.

dtolnay commented 5 years ago

If the intention is to make parsing easier, that doesn't require <...> to be a delimiter. For example representing it as a None-delimited group inside of which the first and last token are < and > would work just as well.

petrochenkov commented 5 years ago

I thought about this, but assumed that there's a difference between a "proper group" and "None-delimited group that just happens to start/end with angle brackets". But perhaps that's not actually so important? Hmm.

dtolnay commented 5 years ago

I believe those would be equivalently easy for a parser to handle (at least Syn's parser).

alexcrichton commented 5 years ago

Removing nomination due to discussion in libs triage, @dtolnay will follow-up with some more specifics

petrochenkov commented 5 years ago

I'm still interested in what @dtolnay was planning to write, but I'm going to close this issue because balanced angle brackets turned out to not be an acceptable solution for the turbofish problem (https://github.com/rust-lang/rfcs/pull/2544#issuecomment-423827385).