Deserialize incomplete message

[GoogleCodeExporter]

Please include an e-mail address if this might need a dialogue!
==============

What steps will reproduce the problem?

1. initialize two communicating TCP clients: A and B

2. write a truncated message (byte array) to the stream of A (message -> 
byte array, truncate the array, write truncated array to stream of TCP 
client)

3. write a non-truncated message to the stream of A 
(SerializeWithLengthPrefix)

4. deserialize the incoming messages on B (DeserializeWithLengthPrefix)

What is the expected output? What do you see instead?

B blocks after receiving the truncated message and fails after receiving 
the non-truncated message. If I'm not mistaken, it seems that B does not 
realize that it received an incomplete message.

What version of the product are you using? On what operating system?

protobuf r262 cf35 on Windows XP

Please provide any additional information below.

Original issue reported on code.google.com by bart.deb...@gmail.com on 28 Aug 2009 at 12:33

[GoogleCodeExporter]

I'm not entirely sure how you would prefer it to work, at least with raw 
sockets... if 
you haven't closed the stream then blocking (when you've advertised 80 bytes 
and sent 
65) is probably expected... how would you have this work?

Original comment by marc.gravell on 28 Aug 2009 at 6:04

[GoogleCodeExporter]

Dear Marc,

  Thanks for coming back on this issue!

  Blocking after receiving an incomplete message seems reasonable to me if 
DeserializeWithLengthPrefix first returns null when the second non-truncated 
message 
arrives (to indicate the truncated message) and then returns the deserialized 
non-
truncated message.

  Does that sound reasonable?

Thanks,
Bart

Original comment by bart.deb...@gmail.com on 28 Aug 2009 at 11:09

[GoogleCodeExporter]

How could it possibly know... the assumption is that the data stream itself 
will be 
intact; if you need additional security around that then you would have to wrap 
it 
inside whatever additional transport layers you choose to use. If you are just 
using 
raw streams, then /n/ bytes is /n/ bytes...

Original comment by marc.gravell on 28 Aug 2009 at 11:37

[GoogleCodeExporter]

If the assumption is that the data stream is intact, it can not be used to 
reliable 
transfer data: tcp/ip connections over a wireless network do lose packets ...

Does protobuf have unique tags that can be used to mark to start of a message?

thanks,
Bart

Original comment by bart.deb...@gmail.com on 28 Aug 2009 at 12:14

[GoogleCodeExporter]

Protobuf only defines the data specification; it doesn't provide or define 
anything 
*other* than the data. If you need a strategy for packing this data, then 
perhaps embed 
the protobuf stream *inside* whichever other transport extensions you need to 
guarantee 
integrity. But the Google spec simply has no provision for this. If this is a 
problem 
(are you using UDP?) then perhaps pre-buffer your data, so you know you have a 
valid 
message before processing it.

Marc

Original comment by marc.gravell on 28 Aug 2009 at 7:25