Protobuf segfaults on attempt to serialize a very large object

[GoogleCodeExporter]

What steps will reproduce the problem?
1. attempt to serialize message larger than 2.1 Gb
2. Running on RHEL 5. using gcc 4.1.2.
3.

What is the expected output? What do you see instead?
  Expect serialization to complete. Instead getting a segfault.

What version of the product are you using? On what operating system?
  2.3.0. 
  Checked source code for 2.5.0, which appears to have same issue.

Please provide any additional information below.

problem appears to be in "bool StringOutputStream::Next(void** data, int* 
size)" implementation.

please see suggested patch in attachment.

Problem occurs when size of serialized object grows above 2.1 Gb (2 ^ 31 
bytes). In current implementation there is a signed integer that holds "old 
size":
  int old_size = target_->size();
  ...
  max(old_size * 2,
      kMinimumSize + 0));  // "+ 0" works around GCC4 weirdness.

Thus it is possible that "old_size * 2" becomes negative. And as a result of 
that "new size" becomes actually less than original one, which eventually leads 
to segfault (on attempt to access 'old_size' element in reallocated small 
buffer).

It is a separate question on whether or not it is good design that leads to 
such large serialized objects, but I don believe that protobuf library code 
should cause segfault.

This actually brings a related question on signature of following methods:
   bool Next(void** data, int* size)
   void BackUp(int count)
   int Skip(int count)

Should not "size" argument be unsigned?

Original issue reported on code.google.com by leonid.g...@gmail.com on 13 May 2014 at 3:55

Attachments:

• zero_copy_stream_impl_lite.cc.patch

[GoogleCodeExporter]

failed to mention - I am just reporting this bug.
Both discovery of bug and solution was done by Andrey Pliss.

Original comment by leonid.g...@gmail.com on 13 May 2014 at 3:57

[GoogleCodeExporter]

Protobuf uses int to represent sizes so the largest size it can possibly 
support is <2G. We don't have any plan to change int to size_t in the code. 
Users should avoid using overly large messages.

Original comment by xiaof...@google.com on 13 May 2014 at 5:29