search

zyq001 / thrift-protobuf-compare

Automatically exported from code.google.com/p/thrift-protobuf-compare
0 stars 0 forks source link

Deserialize and Check timings are incorrect for plain ObjectSerializers. #22

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Tests that implement plain ObjectSerializer have an unfair advantage over
tests that implement CheckingObjectSerializer.  Even for non-lazy parsers,
doing the field checks takes a significant amount of time.  BenchmarkRunner
assumes that, for ObjectSerializers, this time is zero.

Currently, Protobuf is getting screwed by this (~30% time increase on the
"Deserialization Time" graph on the "Benchmarking" page).  Most other
non-lazy parser tests do not implement CheckingObjectSerializer.

Two way to fix this problem:
1. Convert all the tests to CheckingObjectSerializer.
2. Not report "Deserialize and Check" times for plain ObjectSerializer tests.

A quick way to do (2) is to go to BenchmarkRunner.java, lines 259-260 and
replace them with:

   double timeDeserializeAndCheckAllFields = -1;
   double timeDeserializeAndCheckMediaField = -1;

This'll cause "-1" to appear in the output.  No, it's not pretty, but at
least the results are no longer misleading.  I think it's a good temporary
fix until someone puts in the time to do (1) or something better.

Original issue reported on code.google.com by kannan%c...@gtempaccount.com on 22 Feb 2010 at 6:20