search

yav / haskell-lexer

A fully compliant Haskell 98 lexer.
MIT License
15 stars 7 forks source link

Parsing of line comment fails if there is no trailing LF (but rather EOF). #9

Open phadej opened 3 years ago

phadej commented 3 years ago

Compare

no LF

*CabalDocspec.Lexer> traverse_ print $ L.lexerPass0 "1 + 1 -- foo"
(IntLit,(Pos {char = 0, line = 1, column = 1},"1"))
(Whitespace,(Pos {char = 1, line = 1, column = 2}," "))
(Varsym,(Pos {char = 2, line = 1, column = 3},"+"))
(Whitespace,(Pos {char = 3, line = 1, column = 4}," "))
(IntLit,(Pos {char = 4, line = 1, column = 5},"1"))
(Whitespace,(Pos {char = 5, line = 1, column = 6}," "))
(Commentstart,(Pos {char = 6, line = 1, column = 7},"--"))
(ErrorToken,(Pos {char = 8, line = 1, column = 9}," foo"))
(GotEOF,(Pos {char = 12, line = 1, column = 13},""))

With LF

*CabalDocspec.Lexer> traverse_ print $ L.lexerPass0 "1 + 1 -- foo\n"
(IntLit,(Pos {char = 0, line = 1, column = 1},"1"))
(Whitespace,(Pos {char = 1, line = 1, column = 2}," "))
(Varsym,(Pos {char = 2, line = 1, column = 3},"+"))
(Whitespace,(Pos {char = 3, line = 1, column = 4}," "))
(IntLit,(Pos {char = 4, line = 1, column = 5},"1"))
(Whitespace,(Pos {char = 5, line = 1, column = 6}," "))
(Commentstart,(Pos {char = 6, line = 1, column = 7},"--"))
(Comment,(Pos {char = 8, line = 1, column = 9}," foo\n"))