rune-rs / rune

An embeddable dynamic programming language for Rust.
https://rune-rs.github.io
Apache License 2.0
1.65k stars 84 forks source link

Fix quote! parsing of punctuations such as `?;` #580

Open ModProg opened 1 year ago

ModProg commented 1 year ago

Probably quote shouldn't parse any joint punctuation as one token.

udoprog commented 1 year ago

Could you elaborate? I don't understand what this issue relates to.

ModProg commented 1 year ago

quote!(something?;) throws this error.

udoprog commented 1 year ago

Can you reproduce the exact error here for reference?

ModProg commented 1 year ago
error: unsupported punctuation
  --> src/env.rs:47:30
   |
47 |         rune::macros::quote!(?;);
   |                              ^
ModProg commented 1 year ago

I expect the reason is that quote parses all punctuation that is Spacing::Joint as one tt, but it should be two in this case.

udoprog commented 1 year ago

Right, so yeah, it's an error we raise: https://github.com/rune-rs/rune/blob/40786fe4ec1f004cb443146032fbade9974111ca/crates/rune-macros/src/quote.rs#L115 So we need to fix it, somehow :)

udoprog commented 1 year ago

So a way to do this would be to improve kind_from_punct to process more than one punctuation in case it's a joint composite which can actually be decomposed into multiple punctuations. Since both ? and ; are already separate punctuations.

ModProg commented 1 year ago

I have a list of the conditions to parse rust tts from joint tokens here: https://github.com/ModProg/proc-macro-utils/blob/196525b066a3b5f8d751da6e7574731790827bb3/src/parser.rs#L1047