zesterer / chumsky

Write expressive, high-performance parsers with ease.
https://crates.io/crates/chumsky
MIT License
3.63k stars 155 forks source link

Add a new `ParserExtra::State: Inspector` trait #681

Closed jyn514 closed 1 month ago

jyn514 commented 1 month ago

This allows using imperative concrete syntax tree parsers like rowan and cstree. In particular those libraries want to know every token that is parsed, and need to know when chumsky backtracks and reparses the same tokens again.

This adds the following new API surface:

pub trait Inspector<'a, I: Input<'a>>: Default {
    type SaveMarker: Copy + Clone;
    fn on_token(&mut self, token: &I::Token);
    fn on_save<'parse>(&self, offset: I::Offset) -> Self::SaveMarker;
    fn on_rewind<'parse>(&mut self, marker: Marker<'a, 'parse, I, Self::SaveMarker>);
}
pub struct SimpleState<T>(pub T);
impl<'a, T, I: Input<'a>> Inspector<'a, I> for SimpleState<T>;
impl<T> DerefMut<Target = T> for SimpleState<T>;
impl<T> From<T> for SimpleState<T>;

and additionally now requires ParserExtra::State: Inspector.

i didn't write any of my own tests yet but my own parser that hooks chumsky up to cstree does work: https://github.com/spreadsheet-lang/spreadsheet/commit/1b9731311aa97c1130c76de67966052909c35f75#diff-14962dfdd56397d99e07041d6337cb5b0327f4d7433d80fd3383955415014910R111

Fixes https://github.com/zesterer/chumsky/issues/96

zesterer commented 1 month ago

Looks like there are a few minor CI failures

zesterer commented 1 month ago

Thanks so much for your work on this, it's very appreciated!