protobuf-php / protobuf

PHP implementation of Google's Protocol Buffers
MIT License
236 stars 29 forks source link

Add support for message delimiting #13

Open ghost opened 8 years ago

ghost commented 8 years ago

Google's Java implementation of protobuf provides the writeDelimitedTo/parseDelimitedFrom methods, basically just prefixing individual messages with their length and thus allowing multiple messages to be combined in one output stream.

I've been trying to cobble this together myself, but I think it would be nice to have as part of the library as well. This would make API creation using your library much easier, since you might wanna support requests querying multiple resources/messages of one kind.

If you're not likely to implement this, could you maybe point me in the right direction for doing this myself? It's no problem for small messages (where the length fits into 8 bytes)

pack("c*", strlen($message->toStream()));

but for larger messages, packing with v* or V* produces errors on the client's side, because their length needs to be converted to a varint.

Mararok commented 8 years ago

Hey you don`t need do 'strlen' on Stream. You can use Stream::getSize() and prefix message with that value. If your message is very tiny you need only 1 or 2 bytes for encode size.

Write:

$stream = $message->toStream();
$packedSize = pack('n', $stream->getSize());
$data = $packedSize.$stream->getContents();
fwrite($file, $data);

Read:

$packedSize = fread($file, 2);
$size = unpack('n' $packedSize)[1];
$messageData = fread($file, $size);
$message = new YourMessage(Stream::fromString($messageData));

And its good idea to add that functionality to lib.

FabioBatSilva commented 8 years ago

You can probably use YourMessage#writeTo() and YourMessage#serializedSize() to write multiple messages to the same Stream.

Implementing writeDelimitedTo/parseDelimitedFrom would require changes to protobuf-php/protobuf-plugin to generate the new methods or if possible implementing it in AbstractMessage

But given a good PR I would be happy to reconsider it.