Performance and threading patch

[GoogleCodeExporter]

Here are some possible changes that address threading and performance issues.

The course locking on both Btree and CacheRecordManager have been reduced.

The CacheRecordManager change also reenables a bounded soft cache or falls back 
to a weak cache.

Original issue reported on code.google.com by federate...@gmail.com on 22 Jul 2010 at 7:23

Attachments:

• jdbm.patch

[GoogleCodeExporter]

Is there any interest in the supplied patch?

Just to explain more:

Full synchronization of the CacheRecordManager unnecessarily blocks cache 
operations when the underlying RecordManager is in use.

The changes to BaseRecordManager aren't yet needed as the use of bufferInUse is 
fully synchronized, so it could just as easily be non-volatile.  I was going to 
take a pass at reducing the locking there though as well, if there is interest.

I did see that the soft cache was renabled in a later check in, but it is still 
unbounded.  That requires a separate thread and may not perform well with high 
volumes of data.

Original comment by federate...@gmail.com on 29 Jul 2010 at 8:42

[GoogleCodeExporter]

The coarse grained method level sync blocks of JDBM 1.x have been a performance 
issue for me. I'd be interested in the functionality of this patch making it 
into the project. 

Can the devs update us on the latest plans for concurrency support for JDBM 
2.0? I read in the announce thread for this project on the old SF site that 
concurrency improvements would be part of the work.

Original comment by elstenso...@googlemail.com on 6 Aug 2010 at 8:26

[GoogleCodeExporter]

Hi,
sorry for a delay (holidays) and thanks for reading source code.

I will have look at this patch. CacheRecordManager improvement could be 
accepted, but I need to check that RecordManager does not modify instances 
which are already in cache. 

In general I prefer to use 'big lock', it makes it simple and I dont have time 
to fix concurrency issues. Elias Ross was working on concurrency before, but I 
did not integrate his work into JDBM2. 

Original comment by openco...@gmail.com on 12 Oct 2010 at 12:26

[GoogleCodeExporter]

The Hadoop-Hive guys have reported a problem with the old JDBM. Sadly they've 
stopped using it since - 
http://wiki.apache.org/hadoop/Hive/JoinOptimization#A1.2_Removing_JDBM

FYI.

Original comment by ashwin.j...@gmail.com on 3 Dec 2010 at 4:21

[GoogleCodeExporter]

I was unable to reproduce problem reported by Hadoop guys. 

This patch is not yet in JDBM 2.0, I will add it in next version

Original comment by kja...@gmail.com on 25 Dec 2010 at 9:00

[GoogleCodeExporter]

I started integrating concurrency cache patch. Will be in JDBM 2.1

Original comment by kja...@gmail.com on 22 Jan 2011 at 5:35

• Changed state: Started

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I've had a bash at this as well. My code is in issue #4.

http://code.google.com/p/jdbm2/issues/detail?id=4

Original comment by pmurray....@gmail.com on 14 Mar 2011 at 5:59

[GoogleCodeExporter]

Hi, 
I integrated patch into trunk. Thanks for improving JDBM and sorry for long 
delay.

Original comment by kja...@gmail.com on 14 Apr 2011 at 6:49

• Changed state: Fixed