Closed alan-j-hu closed 2 years ago
I think we can do this. I quickly tried to modify a lexgen-generated lexer to add a user state with a lifetime. We should parse the lifetime parameters in user state and add those to the generated lexer struct after the 'input
lifetime.
For example, for a lexer definition like
lexer! {
Lexer(LexerState) -> Token;
...
}
We generate this struct:
struct Lexer<'input, I: Iterator<Item = char> + Clone>(
::lexgen_util::Lexer<'input, I, Token, UserState, ::std::convert::Infallible, Lexer<'input, I>>,
);
If we allow lifetime parameters in LexerState
we should parse those and add them to the struct:
lexer! {
Lexer(LexerState<'a>) -> Token;
...
}
==>
struct Lexer<'input, 'a, I: Iterator<Item = char> + Clone>(
::lexgen_util::Lexer<'input, I, Token, UserState<'a>, ::std::convert::Infallible, Lexer<'input, 'a, I>>,
);
Same when generating impl
blocks.
I think that's it. Shouldn't be too difficult. If anyone wants to implement this I'm happy to help with the code. If not, I can give it a try, but possibly in ~2 weeks. (I'll be mostly offline next week)
Currently, the only lifetime parameter generated is
'input
(with the hardcoded name): https://github.com/osa1/lexgen/blob/main/crates/lexgen/src/dfa/codegen.rs#L191I would like to create a state type with another lifetime, i.e.
pub Lexer(State<'a>) -> Token<'input>;
, but this isn't allowed.Thank you for your time and consideration!