ncupper / protobuf-net

Automatically exported from code.google.com/p/protobuf-net
Other
0 stars 0 forks source link

Deserializing big files (>2GB) end up in an exception #384

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. Make a big list of Objects
2. Every Object contains two big list of random integers
3. Serialize this data structue, so that the file size is > 2 GB
4. Deserialize it

What is the expected output? 
I expect the list of objects containing two lists of integers.

What do you see instead?
An exception.

What version of the product are you using? On what operating system?
v2.0.50727

Original issue reported on code.google.com by xxxditze...@gmail.com on 2 Jun 2013 at 5:21

GoogleCodeExporter commented 9 years ago
The message is: "Sub-message not read correctly"

Original comment by xxxditze...@gmail.com on 2 Jun 2013 at 5:23

GoogleCodeExporter commented 9 years ago
We're sporadically getting the same error message ("ProtoBuf.ProtoException: 
Sub-message not read correctly") for messages of size no more than 7660 bytes, 
and we have no idea why. We recently upgraded from r640 to r666 and it may or 
may not be related. What exactly does this exception mean?

Original comment by allon.guralnek on 30 Sep 2013 at 10:10

GoogleCodeExporter commented 9 years ago
For the 7660 byte scenario I'm guessing this is "after a period of usage"; 
please update to r668 and see if it still happens.

For the > 2GB case - due the bug (introduced r663, now fixed in r668) I know 
exactly what the problem is, and will fix in a future build.

Original comment by marc.gravell on 30 Sep 2013 at 10:13

GoogleCodeExporter commented 9 years ago
It seems that r688 solved all the recent issues we had, both "Sub-message not 
read correctly" and the numerous "No parameterless constructor found for 
<class>" that followed and prevent the application from functioning anymore 
until it was restarted. It usually occurred about two hours after the 
application was started, but now the application is running for far longer 
without an issue. Thanks!

Original comment by allon.guralnek on 30 Sep 2013 at 3:50