search

junneyang / zumastor

Automatically exported from code.google.com/p/zumastor
0 stars 1 forks source link

Reduce the overhead of btree updates during origin writes (aka "leaf node optimization") #106

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
The fix of this issue follows the same approach as that for issue 52
http://code.google.com/p/zumastor/issues/detail?id=52

Original issue reported on code.google.com by jiahotc...@gmail.com on 9 Apr 2008 at 5:09

GoogleCodeExporter commented 9 years ago

Original comment by jiahotc...@gmail.com on 29 Apr 2008 at 1:32

GoogleCodeExporter commented 9 years ago 
A new issue is entered as the first step of this optimization and targeted to 
0.9,
see issue 136. This issue is deferred to 0.10.

Original comment by jiahotc...@gmail.com on 15 May 2008 at 10:35

GoogleCodeExporter commented 9 years ago

Original comment by daniel.r...@gmail.com on 15 May 2008 at 11:09

GoogleCodeExporter commented 9 years ago 
I think you mean issue 139.

Original comment by daniel.r...@gmail.com on 17 May 2008 at 2:05

GoogleCodeExporter commented 9 years ago 
Right. Thanks for correcting it.

Original comment by jiayin...@gmail.com on 17 May 2008 at 2:09

GoogleCodeExporter commented 9 years ago 
Here is a prototype implementation of btree update optimization. 

Rather than optimizing all kinds of btree updates, the patch only optimizes the
journaling of dirty buffers generated during add_exception by recording the
add_exception change in the commit block instead of writing those dirty buffers 
to
the journal. According to my measurement, dirty buffers generated during
add_exception are more than 90% of the total dirty buffers (not including bitmap
dirty buffers already saved by Daniel's bitmap optimization) during origin 
writes. So
this approach should provide comparable performance as optimizing all kinds of 
btree
updates, with the benefit of reduced complexity. 

Basically, before entering add_exception update, we record the variables 
required for
this change in ddsnap superblock. In commit_transaction, we copy the recorded
add_exception changes to commit block. All of the dirty buffers generated during
those add_exception changes are skipped during journal writes. Two fields are 
added
to the ddsnap superblock: record_added_exceptions is the number of recorded
add_exception updates and added_exceptions is the array of recorded 
add_exception
updates. An unsigned field is added to buffer structure to record the sequence 
number
of add_exception changes. I also extended commit_block structure to cover two 
kinds
of journaled change: the normal buffered journal change and the encoded 
add_exception
change. During journal replay, these changes are scanned and recovered 
accordingly. 
I moved replay_journal backward because it needs to call add_exception_to_tree 
now.
The optimization is turned off by default and can be enabled with the
--experimental|-X flag.

Patch is attached. Another attachment is the performance data measured on my 
dell
workstation with nsnaps benchmark and 16k chunksize. In the graph, 'native' is 
the
performance measured without experimental flag, 'experimental' is the 
performance
measured with the current trunk code and with the experimental flag set (i.e., 
bitmap
optimization + deferred journal write optimization), and
'experimental+optimized_btree' is the performance with the patch applied (i.e.,
bitmap optimization + deferred journal write optimization + add_exception 
optimization).

I got network outage problem with the uml cbtb tests. So it hasn't passed all 
of the
uml tests. I will give it another try tomorrow.

Original comment by jiahotc...@gmail.com on 11 Jul 2008 at 2:07

Attachments: