Open mostlymaxi opened 1 year ago
Ah I finally understood what happens: when it reaches the unexpected EOF, the Decoder
returns an error. If you call it again, it will keep returning this same error.
The Lines
iterator will simply forward these errors. Calling .next()
will return an error every time.
As you can see, the only way for Lines
to end iteration is for the reader to return Ok(0)
:
If you unwrap()
or return any error coming from rdr.lines()
, it will stop iteration as expected.
The issue is that the Decoder
doesn't "fuse" (and starts pretending to be an empty stream) when if finds an error.
I wonder how other Reader
implementations react - does a socket pretend to be valid and empty after returning an error if it gets disconnected? :shrug:
Stream decoder will loop forever on an incomplete frame error.
I run into corrupted zst files quite often in life that contain a premature end read error:
test.zst : 1036 MB... test.zst : Read error (39) : premature end
This tends to occur when a zst file is truncated.
I am using the stream decoder to iterate over lines of this corrupted file and the error that I get is this:
Custom { kind: UnexpectedEof, error: "incomplete frame", }
Strangely, this will then try to keep pulling the same frame and loop this error. Should the stream close on error?
Code: