Marwes / combine

A parser combinator library for Rust
https://docs.rs/combine/*/combine/
MIT License
1.29k stars 93 forks source link

Is there a way to get `Stream<Token=char>` from `io::Read`? #329

Open tailhook opened 2 years ago

tailhook commented 2 years ago

Hi!

I'm seeing examples of how to parse and decode bytes from io::Read, but still not clear is now to do the same to have char stream (and also preferably with byte positioner). I find using chars easier for some use cases. Is there any built-in or easily accessible way to do that, or do I have to implement custom stream for that?

Marwes commented 2 years ago

The Read stream https://github.com/Marwes/combine/blob/fdd63e2a8635c00c5c2fe1d18110958178376310/src/stream/read.rs#L157-L175 uses https://doc.rust-lang.org/std/io/trait.Read.html#method.bytes internally however there is no equivalent for reading char directly from an io::Read so you would need to implement that first.

If at all possible I would recommend against parsing directly from a io::Read as it is significantly slower than reading from an in memory buffer. If you need streaming you may want to look into using https://docs.rs/combine/4.6.2/combine/macro.decode.html and partial parsing instead, it may be easier to create an input stream that contains a &[u8], yet returns char/&str (might be a good addition to combine itself actually).