search

haskell / parsec

A monadic parser combinator library
https://hackage.haskell.org/package/parsec
Other
842 stars 95 forks source link

It's pretty hard to make a lexer using Data.Text #59

Open burz opened 7 years ago

burz commented 7 years ago

Although the parser is rather easy to use to export Text from Data.Text, the Text.Parsec.Token library is rather hard to use to create a lexer using Text.

I'm not exactly sure how to make the change as of yet -- I'd love to be guided into making the change -- but it would be wonderful if we could somehow parameterize either makeTokenParser or emptyDef so that it would be easier to create a lexer that can handle to the Text type rather than the String from Prelude.

I would love for someone to correct me if I'm wrong and this is doable with the current libary, but at the same time I feel that it could potentially be a reason someone might look somewhere else. Perhaps having an easy way of creating a Text consuming lexer could be a boon to Parsec.

mrkkrp commented 7 years ago

See mrkkrp/megaparsec#106 for a similar discussion (I think it's applicable to Parsec too).