Protobuf serialization is very slow

[GoogleCodeExporter]

Hi,
This is my first time using Protobuf.NET and I am experiencing significant
delay while serializing a slightly complex structure of mine. Other teams
in my organization have been using protobuf (C++ and Java) and they
recomended it to me so that I can incorporate it in one of the solution I
am working with. I think it is some kind of bug in the implementation. When
I try to serialize similar data structure using XmlSerlialization, results
are promossing but doing the same with protobuf.net is 10 times slower.

I am attaching an application that can simulate the problem at hand and may
be help you with debugging as well. Please try to serialize 50 messages
using both techniques and you will see the difference immediately. It is
the call to ProtoBuf.Serializer.Serialize that is causing the delay.

Regards.

Original issue reported on code.google.com by rehan.na...@gmail.com on 22 Apr 2010 at 3:24

Attachments:

• ProtobufProblem.zip

[GoogleCodeExporter]

Downloaded to investigate. Can you clarify which version you are using? Is this 
the 
downloadable dll? Or have you compiled protobuf-net from the trunk?

Original comment by marc.gravell on 22 Apr 2010 at 3:36

[GoogleCodeExporter]

The short version is: your test has an error. The two button presses act on 
**vastly** different amounts of data.

button1:

                List<AutoGenerated.FPMTR> tempTnrList = new 
List<AutoGenerated.FPMTR>();
...
                msg.Tr = tempTnrList.ToArray();

button2:

                List<PROTO.Tn> tempTnrList = msg.Tns;

This has the impact that in button 1 you create lots of messages, **each of a 
similar 
size**. In button 2
you create **increasingly large** messages. If we change button2 to be 
comparable, by 
adding:

                msg.Tns.Clear();

(or any other mechanism) then I get protobuf-net significantly faster. Bumping 
up the 
numbers to 5000 to make
it obvious gives me:

XML Messages: 18,610
Protobuf messages: 3,329

(note I'm using the pre-release v2 code, so you might see slightly different 
numbers, 
but the same pattern)

Switching to the "v1" dll you included, and re-running the test, I get:

XML Messages: 16,796
Protobuf messages: 6,726

so even if we allow a generous amount of time to allow for variance in my 
unscientific test, we can also
see that "v2" (the not-yet-released version) makes a **significant** further 
improvement on the time.

You should also note that treating dates as strings will be an expensive way of 
doign 
it. If possible, use either a
large integer (ticks in an epoch), or let DateTime serialize itself (although 
the 
format that protobuf-net uses
is a little tricky to digest from non-.NET clients; maybe I'll offer additional 
options there in "v2")

Original comment by marc.gravell on 22 Apr 2010 at 4:36

[GoogleCodeExporter]

Hi,
I downloaded and tried out 2 versions from the downloads section, so no protobuf
compilation on my machine etc. I downloaded the following 2 versions

1- protobuf-net-VS10.msi  Visual Studio 2008 / 2010 support    Apr 14
2- protobuf-net r282.zip  protobuf-net r282    Jan 28

Original comment by rehan.na...@gmail.com on 22 Apr 2010 at 4:37

[GoogleCodeExporter]

My bad, it is fast. Cleaning up the collection every time makes protobuf 
perform as
accepted.
Once again, sorry for disturbance, I should have noticed this myself.

Original comment by rehan.na...@gmail.com on 22 Apr 2010 at 4:41

[GoogleCodeExporter]

Not a problem. I should stress also that "v2" has much richer abilities to do 
this 
without requiring 2 different object models (although most of that is possible 
in "v1" 
too).

Original comment by marc.gravell on 22 Apr 2010 at 6:38

• Changed state: Invalid