gwenn / lemon-rs

LALR(1) parser generator for Rust based on Lemon + SQL parser
The Unlicense
48 stars 10 forks source link

Drop support of buf_redux / streaming input #26

Closed gwenn closed 1 year ago

gwenn commented 1 year ago

Should fix #21 and #23

gwenn commented 1 year ago

Some rustdoc is not up to date like here but I will fix them later...

MarinPostma commented 1 year ago

Actually, there is a drawback to this, the scanner is not allowed to own its buffer. This is an issue when you want to store the parser

MarinPostma commented 1 year ago

I think we should let the scanner own the buffer and add store an offset to the current byte position, instead of re-slicing.

The buffer can be made generic over T: AsRef<[u8]>

gwenn commented 1 year ago

I think we should let the scanner own the buffer and add store an offset to the current byte position, instead of re-slicing.

This solution should also help fixing #4

gwenn commented 1 year ago

With input: AsRef<[u8]>:

error[E0502]: cannot borrow `*self` as mutable because it is also borrowed as immutable
   --> src/lexer/scan.rs:115:25
    |
98  |     pub fn scan(&mut self) -> ScanResult<'_, S::TokenType, S::Error> {
    |                 - let's call the lifetime of this reference `'1`
...
104 |                 let data = &self.input.as_ref()[self.offset..];
    |                             ------------------- immutable borrow occurs here
...
115 |                         self.consume(amt);
    |                         ^^^^^^^^^^^^^^^^^ mutable borrow occurs here
...
120 |                         return Ok(tok);
    |                                ------- returning this value requires that `self.input` is borrowed for `'1`

I should be able to fix consume but not the lifetime issue...

gwenn commented 1 year ago

I think we should let the scanner own the buffer and add store an offset to the current byte position, instead of re-slicing.

I had to do the contrary in 611b94f: the input is now passed as a parameter to the scan method and the parser now keeps a reference to the input (to make the borrow checker happy in Parser::next).