search

travissimon / ghaml

Ghaml is a go parser for a haml-like language. It seeks to provide an efficient alternative to the inbuilt template system.
GNU General Public License v2.0
21 stars 4 forks source link

Q. Lexer using circular buffer? #2

Open rickb777 opened 10 years ago

rickb777 commented 10 years ago

The lexer refers to Rob Pike's excellent talk (http://blog.golang.org/2011/09/two-go-talks-lexical-scanning-in-go-and.html). I am somewhat confused because this builds up a design using a channel and goroutine ... then in the last slide switches to an inline-design without the goroutine.

Instead it adds buffering to the channel and uses it as a circular buffer. This revised strategy is used by Ghaml's lexer. The circular buffer design is probably fine, provided the buffer is big enough - the necessary size for all possible code paths must be predetermined.

What puzzles me is simply this: why does Go's own text/template lexer use the original design with a goroutine (http://golang.org/src/pkg/text/template/parse/lex.go line 191)? Is there a good reason to prefer the concurrent design over the circular-buffer? Or vice versa?

travissimon commented 10 years ago

There are issues with running the lexer as a channel. This code was written a while ago, and without being able to play with it, I might get the exact details wrong. But, I believe it becomes difficult to know when parsing is complete - the channel just stays open, but the consumer doesn't know that lexing is done. I just remember having issues (and yes, my description of the problem does not sound convincing), and they were easily solved with with making a circular buffer.