Remove block comments

[brendanzab]

PR #32 introduced block comments to the lexer. It would be nice if these nested too! This would then admit the following string:

{- 
    {- ... -} 
    {- 
       {- ... -} 
    -}
-}

---

We have decided instead to remove the block comments entirely!

[brendanzab]

Not sure the best approach to this. Perhaps we need a 'seen bracket' counter that we increment and decrement in the lexer?

Or perhaps (as I mentioned in gitter) we could just remove block comments! This is what Futhark has done.

[adrianwong]

I'm currently writing up a Brainfuck interpreter in Haskell, and this was the exact same issue I ran into.

If we take the approach of incrementing a counter on an open brace, and decrementing it on a close brace (and expect the end value to be 0), we don't account for the fact that this case {- -} -} {- will be considered valid.

What I eventually ended up doing is make use of a stack to keep track of the braces:

• Push a token onto the stack on an open brace
• Pop the topmost token on a close brace
• If we pop an empty stack, raise an error
• If the stack is non-empty at the end of the lexing process, raise an error

This takes care of the cases that the counter doesn't.

I'll try to put forward a PR and you can decide if it's really too messy to be worth it. But I think I prefer the idea of removing block comments (or at the very least, not nesting them).

[brendanzab]

Let's just remove them then! Sorry for the wasted effort - I guess it helped to get an idea of what the lexer looks like though, hopefully.

[adrianwong]

Nah, all good! Yeah it was quite informative. Also, by attempting to hack in an implementation for this, I got to learn a bit about Option and Vec and pattern matching in Rust.

If you'd like, I can submit a PR to remove support for block comments.

[brendanzab]

Sounds great! Yeah, it's not too dissimilar to Maybe in Haskell. Vec is like std::vector in C++.

[adrianwong]

I just realised that my post about using a stack in favour of a counter makes no sense... We could just raise an error if we're decrementing when the counter is 0.

I was fairly certain that there was an edge case which I was accounting for, but now I feel a bit foolish...

[brendanzab]

Easy mistake to make if your nose is right next to the problem! 🙂