kevinmehall / rust-peg

Parsing Expression Grammar (PEG) parser generator for Rust
https://crates.io/crates/peg
MIT License
1.46k stars 106 forks source link

Reached the type-length limit on parser! macro invocation #226

Closed p0lunin closed 3 years ago

p0lunin commented 4 years ago

I got a very strange error:

error: reached the type-length limit while instantiating `parser::lang::__parse_logic::<[c...]]]]]]]]]]]]]]]]>::{{closure}}#0`
   --> src/parser.rs:97:1
    |
97  | / peg::parser! { grammar lang() for str {
98  | |     pub rule parse_lang() -> Vec<Spanned<TopLevelToken>>
99  | |         = (top_level_token())*
100 | |
...   |
228 | |     rule indentation(i: usize) = [' ']*<{i*4}>
229 | | }}
    | |__^
    |
    = note: consider adding a `#![type_length_limit="1900550"]` attribute to your crate
    = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

when i try cargo build. I tried to create MRE, but if i moved all code related to parsing into another crate, it worked without errors. Build in actions: https://github.com/p0lunin/vglang/runs/663466273 When i build use macro-backtrace, it show:

   ::: C:\Users\Asus\.cargo\registry\src\github.com-1ecc6299db9ec823\peg-macros-0.6.2\lib.rs:21:1
    |
21  | / pub fn parser(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
22  | |     let tokens = tokens::FlatTokenStream::new(input.into());
23  | |     let grammar = match grammar::peg::peg_grammar(&tokens) {
24  | |         Ok(g) => g,
...   |
31  | |     translate::compile_grammar(&grammar).into()
32  | | }
    | |_- in this expansion of `peg::parser!`
    |
    = note: consider adding a `#![type_length_limit="2031621"]` attribute to your crate
p0lunin commented 4 years ago

I find where error:

rule logic(i: usize, single_rule: rule<Token>) -> Token = precedence! {
   ...
   d:block(i, <logic(i, <single_rule()>)>) { d }
}

when i comment these lines with recursive call logic(i, <single_rule()>) build was succesful

p0lunin commented 4 years ago

I fixed this error by adding copy of logic() rule, but i don't like this because i duplicate the same code

kevinmehall commented 4 years ago

The minimized version is:

rule logic(single_rule: rule<()>) = " " logic(<single_rule()>)

This expands to something like

fn logic(single_rule: impl Fn()) {
    logic(|| single_rule())
}

which fails to compile with the same error.

This is because it creates an infinite chain of monomorphized versions of the closure. In plain Rust, you could work around this by passing single_rule instead of || single_rule(), but in rust-peg we prohibit that with "parameter single_rule expects a PEG expression, but a value was passed". That restriction could be loosened to allow passing the existing closure, or we could look at using &Fn closures instead of impl Fn to avoid the monomorphization.

kevinmehall commented 3 years ago

https://github.com/kevinmehall/rust-peg/commit/7b45748b9d5d5969b4f834a70a591937164944ad removes the "parameter single_rule expects a PEG expression, but a value was passed" error, allowing you to pass down the rule closure unmodified as a way to avoid the issues with re-capturing the closure:

rule logic(single_rule: rule<()>) = " " logic(single_rule)