Allow the user state to carry a lifetime?

[alan-j-hu]

Currently, the only lifetime parameter generated is 'input (with the hardcoded name): https://github.com/osa1/lexgen/blob/main/crates/lexgen/src/dfa/codegen.rs#L191

I would like to create a state type with another lifetime, i.e. pub Lexer(State<'a>) -> Token<'input>;, but this isn't allowed.

Thank you for your time and consideration!

[osa1]

I think we can do this. I quickly tried to modify a lexgen-generated lexer to add a user state with a lifetime. We should parse the lifetime parameters in user state and add those to the generated lexer struct after the 'input lifetime.

For example, for a lexer definition like

lexer! {
    Lexer(LexerState) -> Token;
    ...
}

We generate this struct:

struct Lexer<'input, I: Iterator<Item = char> + Clone>(
    ::lexgen_util::Lexer<'input, I, Token, UserState, ::std::convert::Infallible, Lexer<'input, I>>,
);

If we allow lifetime parameters in LexerState we should parse those and add them to the struct:

lexer! {
    Lexer(LexerState<'a>) -> Token;
    ...
}

==>

struct Lexer<'input, 'a, I: Iterator<Item = char> + Clone>(
    ::lexgen_util::Lexer<'input, I, Token, UserState<'a>, ::std::convert::Infallible, Lexer<'input, 'a, I>>,
);

Same when generating impl blocks.

I think that's it. Shouldn't be too difficult. If anyone wants to implement this I'm happy to help with the code. If not, I can give it a try, but possibly in ~2 weeks. (I'll be mostly offline next week)