amphp / byte-stream

A non-blocking stream abstraction for PHP based on Amp.
https://amphp.org/byte-stream
MIT License
367 stars 31 forks source link

Ability to read particular length from stream #78

Closed userqq closed 11 months ago

userqq commented 4 years ago

Hi. I think it's would be great to have possibility to read chunks with specified length since commonly dealing with binary protocols looks like

$lengthBytes = fread($fd, 4); // read Int32 Length
$length = unpack('Llength', $lengthBytes)['length'];
$packet = fread($fd, $length);

Not sure about possibility to make InputStream behave like public function read(?int $length = null): Promise;, but maybe some buffering class similar to LineReader would be useful, for example:

class LengthReader
{
    public function __construct(InputStream $inputStream);

    public function readLine(int $length): Promise;
}
kelunik commented 4 years ago

We didn't include any length in the stream read operation on purpose, because it would complicate all stream implementations and will be less efficient if it didn't buffer internally, due to many small reads.

For our Http2Parser we use a custom function, but it also uses a \Generator based parser instead of using promises. For simply uses a wrapper class could also directly offer readInt32 and similar methods.

I think we already have such an implementation somewhere, I just don't remember where.

danog commented 4 years ago

I created my own buffered stream implementation for MadelineProto, still gotta split it from the main project aaaa

userqq commented 4 years ago

I would like to suggest something like following:


class BinaryBuffer
{
    public function __construct(InputStream $inputStream);

    public function read(int $length) : string

    public function unpack(array $format) : Promise<array>
}

Which is inpired by https://github.com/clue/php-socks/blob/a9cd68376de2edb97278dbefc994f679e9360248/Socks/StreamReader.php#L19-L42

And it could be used like:

$buffer = new BinaryBuffer($stream);

try {
    yield $buffer->read(4); // some padding bytes
} catch (UnexpectedStreamClosedException $e) {
    // stream was closed unexpectedly
} catch (NotEnoughInputException $e) {
    // stream was ended before 4 bytes were received
}

try {
    ['length => $length, 'magic' => $magic, 'version' => $version] = yield $buffer->unpack([
        'length' => 'L',
        'magic'  => 'C',
        'version' => ['a', 16], // well, I know it's looks not well. need some better idea
    ]);

    [$packet] = yield $buffer->unpack([
        ['a', $length]
    ]);
} catch (UnexpectedStreamClosedException|NotEnoughInputException $e) {
    // the same as before
}
kelunik commented 11 months ago

You can now use BufferedReader in v2.x.