proposed optimization for parsing of packed repeated fields

[GoogleCodeExporter]

What steps will reproduce the problem?
1.Create a message type with a packed repeated field of int32
2.Compile with the protoc compiler...
3.Parse a message with about 300-400 integers (in the packed field)

What is the expected output? What do you see instead?
It works, but could be faster

What version of the product are you using? On what operating system?
2.4.1 on win7 (x64)

Please provide any additional information below.

I've already posted this in the GPB group 
(https://groups.google.com/forum/#!topic/protobuf/P0GBgD74v08), and was 
suggested to open an issue.

basically I have a process that spends about half of its time parsing a 
protobuf message with a large repeated field of integers (300-400 items). 
profiling this process showed that ‘ReadPackedPrimitive’ dominated the 
parsing, a closer look at the code revealed that it calls 
‘BytesUntilLimit()’ proportionally to the number of items in the repeated 
field, this method evaluates a condition and several arithmetic operations 
which seems harmless but become annoying when executed repeatedly.

I initially patched my generated code to use a local function instead of 
‘ReadPackedPrimitive’, the basic idea was to check the size of the 
available buffer before resorting to ‘BytesUntilLimit()’, this buffer is 
usually large enough to hold the next item (actually, in my case the entire 
message feats in buffer). The result of this patch was a 10-15% increase in 
throughput of my process, and 20% increase when the process was 'crippled' to 
do only protobuf parsing (no farther processing).

I'm attaching a patch to wire_format_lite_inl.h, this patch is made on top of 
V2.4.1

I'd appreciate any feed-back, and be more than happy if it's accepted.

Eyal. 

Original issue reported on code.google.com by Eyal.Far...@gmail.com on 25 Oct 2011 at 11:55

Attachments:

• protobuff.packed_rfield.diff

[GoogleCodeExporter]

Thanks for the patch! Could you please sign the Contributor License Agreement, 
which gives Google a license to use your work.  Can you please let me know 
after you've signed one of the following:

http://code.google.com/legal/individual-cla-v1.0.html -- If you own copyright 
on your patch.  (This can be signed via a simple web form at the bottom of the 
page.)
http://code.google.com/legal/corporate-cla-v1.0.html -- If your employer does.

Sorry for the inconvenience.

Original comment by liujisi@google.com on 7 Nov 2011 at 8:52

• Changed state: Accepted

[GoogleCodeExporter]

I'm attaching a signed license (corporate addition).
please move forward with this patch.

thanks,
Eyal.

Original comment by Eyal.Far...@gmail.com on 4 Dec 2011 at 10:02

Attachments:

• doc04122011105455.pdf