search

jankotek / JDBM3

Embedded Key Value Java Database
367 stars 94 forks source link

Possible serialization constructor performance issue. #76

Closed jankotek closed 12 years ago

jankotek commented 12 years ago

Hi,

problem is disabled cache. It causes BTreeNode to be deserialized for each item. With cache enabled serialization should be called 100x less. Basic test to prove it, is to create your own serializer and put counter there.

Anyway, your stack trace points to possible bug. Constructor instance is cached in HashMap<Class,Constructor>. But stack trace suggests it is called too many times, so caching probably fails.

I created new bug for this:

On 05/09/2012 05:34 AM, Fenn wrote:

Hi all,

I've been testing out JDBM3 a little and seem to be experiencing quite poor deserialization performance.

Retrieving ~6000 records via a few map.subMap() calls seems to reliably take >7000ms (with caching disabled). This is on a very small DB, details:

PAGES: 517 used pages with size 2068KB 25 record translation pages with size 100KB 0 free (unused) pages with size 0B 24 free (phys) pages with size 96KB 1 free (logical) pages with size 4096B Total number of pages is 567 with size 2268KB RECORDS: Contains 16579 records and 421 free slots. Total space occupied by data is 1888KB Average data size in record is 117B Maximal data size in record is 658B Space wasted in record fragmentation is 21KB Maximal space wasted in single record fragmentation is 231B

jvisualvm reveals that most of the time seems to be spent in a single method, namely: sun.reflect.ReflectionFactory.newConstructorForSerialization()

~25% spent via BTreeSet$1.next(): https://img.skitch.com/20120509-maa1j8mwjaabqpn3ya6jxg1au7.jpg (via BTreeMap.size() oddly)

~20% spent in via BTreeSet.iterator(), also via BTreeMap.size(): https://img.skitch.com/20120509-bwkhmxtw2ciiqsdhf3bj7rdmc.jpg

~25% spent via BTreeMap$2$1.next(): https://img.skitch.com/20120509-da442xxn2efuhpt1x5xh89cxjn.jpg

All up, that's 70% of execution time spent (from what I can gather) getting constructors - I've found a few other threads on other mailing lists mentioning similar performance issues in other projects (ie: https://groups.google.com/d/topic/kryo-users/BoyLEe-jye8/discussion).

Does this seem right? In my case, all the objects are of the same class (which presumably would be the case for most Maps), so it seems redundant to have to create constructors every time.

Is there some way I can structure things so that a specific constructor is called upon deserialization?

Cheers/thanks,

Fenn.

jankotek commented 12 years ago

Serialization constructors are going to be redone in JDBM4, so closing.