Reflection-based array operations are rather expensive

[GoogleCodeExporter]

Latest versions of protostuff-runtime use java.lang.reflect.Array class to 
create arrays and get/set elements at a given index. It is a generic way to 
implement things and works correctly. But according to profiler statistics, 
these operations consume a lot of overall CPU time taken by protostuff.

Therefore, I tried to produce a workaround which would avoid using reflection 
as much as possible. Please find attached a patch that tries to implement this 
functionality. 

Please let me know, if it makes sense and if its worth including into trunk.

Original issue reported on code.google.com by romixlev on 30 Apr 2012 at 4:17

Attachments:

• protostuff-runtime-array-patch-v1

[GoogleCodeExporter]

Perhaps there are other ways to optimize one-dimensional primitive arrays.
We can have that baked-in.

You can also use delegates to serialize them:
See 
http://code.google.com/p/protostuff/source/browse/trunk/protostuff-runtime/src/t
est/java/com/dyuproject/protostuff/runtime/SampleDelegates.java

Original comment by david.yu...@gmail.com on 2 May 2012 at 7:41

[GoogleCodeExporter]

Hmm. Yes, delegates could be the way probably.

But since one-dimensional arrays (primitive and Objects) are used so often, may 
be you should try to apply ArrayDelegate automatically by default on all 
one-dimensional array fields? Could you do it? Or do you think that this is an 
optional optimization?

Original comment by romixlev on 3 May 2012 at 7:27

[GoogleCodeExporter]

A delegates module (or even a third party module you can create) can be used to 
contain common types that the user might want to handpick and register 
individually.

A user might want to encode short[] as varints inside a byte[].  Like our 
previous discussion, it is best that the user have explicit control how each 
type is serialized.

Original comment by david.yu...@gmail.com on 3 May 2012 at 7:50

[GoogleCodeExporter]

I agree that the user should have a total control. But only if he wants.
I.e. some reasonably efficient defaults could be very convenient. 
E.g. if protostuff-runtime by default maps: int[] -> usual ints or varints 
inside a byte[], then it is a good and fast default. 

And if required, the user can still override it by explicitly calling:
registerDelegate(int[].class, myCustomDelegate)

I discuss it because right now, the defaults for Arrays and may be a few more 
data-types are not too efficient.

BTW, Kryo tries to find a best matching, most efficient known delegate/custom 
serializaer automatically. But it allows you as a user to override it.

Original comment by romixlev on 3 May 2012 at 8:13

[GoogleCodeExporter]

I've found out that as well, we deserialize about 20000 bigger objects with 
protostuff and more than 80% of the time (7500ms) is taken by 
java.lang.reflect.Array.set(6200ms). I'll investigate the workaround described 
above, but a more performant default would be great!

Original comment by c...@itscope.de on 3 Dec 2012 at 10:24