stream_reader() hangs (keeps spinning on CPU) when empty buffer is supplied

[piotrdomagalski]

Hi there!

We've run into an issue where bad input data caused the library to keep spinning in decompression_reader.c:370 as seen on the screenshot from py-spy stacktrace dump.

Here's the code to reproduce this situation:

import zstandard
import io

decompressor = zstandard.ZstdDecompressor()
bad_input = b''

with decompressor.stream_reader(bad_input) as decompressing_reader:
    with io.TextIOWrapper(decompressing_reader, encoding='utf-8', newline='\n') as reader:
        for line in reader:
            print(line)

[indygreg]

Confirmed. Only reproduces in C backend. Not Rust nor FFI.

Thanks for the report.

[indygreg]

The bug here is that the C backend implementation of ZstdDecompressionReader.read1() will infinite loop for inputs conforming to the buffer protocol having 0 length. This appeared to evade test coverage (including fuzzing) because we never sent an empty input into the fuzz tests. I'll change that as part of fixing this.

[indygreg]

And adding fuzzing test coverage with empty inputs reveals that other methods also choke. e.g. readinto1() is also buggy.