Serializing directly to io.Writer

[GoogleCodeExporter]

It seems that there is currently no way to serialize protobuf structure 
directly to io.Writer in wire format, though MarshalText function seems to be 
present in two forms - the one accepting io.Writer and the one returning string.

Is this done intentionally? This seem to be a problem in case of large 
structures, because currently the data in wire format should be first converted 
to blob and only then sent to writer, which leads to memory [almost] 
duplication (original protobuf-generated structure + byte slice).

As far as I know, C++/Java versions of protobuf API support writing structures 
directly to ostream/OutputStream.

Maybe I'm missing something and there is no such problem present?

Original issue reported on code.google.com by dpx.infinity@gmail.com on 11 Jan 2013 at 9:32

[GoogleCodeExporter]

Also, how am I supposed to deserialize a message of unknown length when I am, 
say, receiving it via socket? Unmarshal functions all accept only []byte or 
string, which implies that I already should know the size of the message. I 
understand that I can for example send the message size before the message over 
the socket, but still C++/Java APIs contain facilities which do this 
automatically. Why shouldn't it be possible in Go? Or am I misunderstanding 
something?

Original comment by dpx.infinity@gmail.com on 11 Jan 2013 at 9:49

[GoogleCodeExporter]

I have wanted something like this for various reasons, and we have discussed it.

Protocol buffers are not self-delimiting, so it is impossible to unmarshal from 
a stream unless the stream is terminated at the end of the message. So it 
*could* be made to work from an io.Reader, but wiring it directly up to a 
socket isn't going to fly.

Original comment by dsymo...@golang.org on 12 Jan 2013 at 12:03

• Changed state: Accepted
• Added labels: Priority-Low, Type-Enhancement
• Removed labels: Priority-Medium, Type-Defect

[GoogleCodeExporter]

Okay, I understand the problem with Reader. It seems that protobuf really does 
something more complicated in Java, and in C++ reading the message from 
istream, according to docs, just exhausts the stream. Still, having the ability 
to serialize directly to Writer would be great.

Original comment by dpx.infinity@gmail.com on 13 Jan 2013 at 9:37

[GoogleCodeExporter]

This may interest you, but I opened up a code review this afternoon to support 
record-length encoded message streaming: 
https://codereview.appspot.com/9102043/.

Original comment by matt.proud@gmail.com on 1 May 2013 at 8:28