ericvana / protobuf-net

Automatically exported from code.google.com/p/protobuf-net
Other
0 stars 0 forks source link

I think with 100 runs of same code on same data I can see all exception of protobuf-net !! #451

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago

==============

What steps will reproduce the problem?
1. Following problem definition on: 
http://stackoverflow.com/questions/26216755/protobuf-net-asking-for-typemodel-cs
-when-used-with-generics-for-deserialization

Testing +50,000,000 items to serializer/deserialized which are all through Add 
and Update processes to BPlusTree at almost random time I randomly received one 
of the following messages: 

a. Invalid wire-type; this usually means you have over-written a file without 
truncating or setting the length; see http://stackoverflow.com/q/2152978/23354

b. Attempted to read past the end of the stream.

c. Invalid field in source data: 0

Given that I *serializeWithLengthPrefix as: 
Serializer.DeserializeWithLengthPrefix<B<C, M>>(stream, PrefixStyle.Fixed32);
Serializer.SerializeWithLengthPrefix<B<C, M>>(stream, value, 
PrefixStyle.Fixed32);

What version of the product are you using? On what operating system?
2.0.0.668
OS: Windows 7
VS2013

Please provide any additional information below.
- 

Original issue reported on code.google.com by hamed.me...@gmail.com on 7 Oct 2014 at 10:14

GoogleCodeExporter commented 9 years ago
Do you have a repro that actually demonstrates this? Is it possibly that your 
your rig uses concurrency and you are simply using the same thread/data from 
parallel threads?

Original comment by marc.gravell on 7 Oct 2014 at 10:21

GoogleCodeExporter commented 9 years ago
I don't know when I'm gonna see an exception next; but will update this issue 
with details if any raised. 

About concurrency: both from mine and BPlusTree's side, it's a single-threaded 
process. 

I also tried Tags as well; as it suggested by @Marc for some questions on 
stackoverflow, but no improvements for my conditions ! 

@Marc, as you have already checked BPlusTree's source, I'm adding the piece of 
code I use for update, "just in case" 
KeyValueUpdate<C, B<C, M>> updateFunction = delegate(C Key, B<C, M> Value)
{
   B<C, M> block = Value;
   block.lambda.Add(new Lambda<C, M>(tau, interval.metadata));
   return block;
};
di3.TryUpdate(key, updateFunction);

Original comment by hamed.me...@gmail.com on 7 Oct 2014 at 10:39

GoogleCodeExporter commented 9 years ago
What I would really love to see is a complete runnable example where I don't 
need to guess at lots of details, so that I can actually try to repro...

Original comment by marc.gravell on 7 Oct 2014 at 10:43

GoogleCodeExporter commented 9 years ago
hmmm ... OK either I should summarize the code (which then you may not see the 
error :D). OR: I can share full project with you, but to run it, somehow I have 
to pass you ~10GB ENCODE data to be able to execute. What do you suggest ?

BTW, the issue is just raised again (after 3 successful runs) 
https://www.dropbox.com/s/v7t2n1gll3guc0e/exception.txt?dl=0

Original comment by hamed.me...@gmail.com on 7 Oct 2014 at 10:56

GoogleCodeExporter commented 9 years ago
I've an idea: updating serialized messages happens quite often in my scenario 
(<20M/50M ), and disabling update everything "seems" to be working fine. This 
encourages me that my assumption with dynamic messages (i.e., 
http://stackoverflow.com/questions/26223017/length-prefix-with-dynamic-messages-
in-protobuf-net) is true ... do you agree ?

Original comment by hamed.me...@gmail.com on 7 Oct 2014 at 11:56