composewell / streamly

High performance, concurrent functional programming abstractions
https://streamly.composewell.com
Other
856 stars 64 forks source link

Parsers to decode to char from Utf8/Latin1 encode bytestream #1198

Open harendra-kumar opened 3 years ago

harendra-kumar commented 3 years ago

A utf8 char parser can be used to implement the current utf8 decode function in a modular way. Just as we can use an unfold to encode a stream (see https://github.com/composewell/streamly/pull/1055).

In module `Streamly.Internal.Unicode.Char

-- Decode a single Char from UTF8 input
parser/char :: Parser m Word8 Char

In module `Streamly.Internal.Unicode.Latin1

-- Decode a single Char from Latin1 input
parser/char :: Parser m Word8 Char
adithyaov commented 2 years ago

There is a parseCharUtf8With now. Keeping this issue open to implement a parseCharLatin1. Currently the parseCharUtf8With in Unicode.Stream, we should probably move it to Unicode.Char