Raku / problem-solving

🦋 Problem Solving, a repo for handling problems that require review, deliberation and possibly debate
Artistic License 2.0
70 stars 16 forks source link

Add `.bufs`/`.blobs` methods to `IO::Path` / `IO::Handle` #420

Open lizmat opened 7 months ago

lizmat commented 7 months ago

Just as .lines provides a lazy Seq of lines, I think it would be worthwhile to have a lazy Seq of buf8/blob8 objects of a file. This could be helpful in some workflows where serially decoding a (large) file is the bottleneck. Using .blobs would give the opportunity to only need to decode those parts one is actually interested in.

Leont commented 7 months ago

But how would it decide how big the blobs should be?

lizmat commented 7 months ago

Whatever IO::Handle.READ returns :-)

Leont commented 7 months ago

Whatever IO::Handle.READ returns :-)

Does that use read(2) or fread(3) semantics? I have a memory of it doing the latter, which would be unusable; I may be mistaken.

Basically, does it retry when getting less than $wanted bytes or not. It truly shouldn't.

lizmat commented 7 months ago

nqp::readfh($!PIO,nqp::create(buf8.^pun),$bytes)

I'm not sure about the semantics, but probably with retry. Not sure what the backends provide.

lizmat commented 7 months ago

FWIW, I think an iterator should use READ semantics.

OTOH, a supply based on asyncreadbytes would be the truly lazy approach.

Leont commented 7 months ago

Basically, does it retry when getting less than $wanted bytes or not. It truly shouldn't.

This is another example of such an issue.