Open GoogleCodeExporter opened 9 years ago
Original comment by jiahotc...@gmail.com
on 29 Apr 2008 at 1:32
A new issue is entered as the first step of this optimization and targeted to
0.9,
see issue 136. This issue is deferred to 0.10.
Original comment by jiahotc...@gmail.com
on 15 May 2008 at 10:35
Original comment by daniel.r...@gmail.com
on 15 May 2008 at 11:09
I think you mean issue 139.
Original comment by daniel.r...@gmail.com
on 17 May 2008 at 2:05
Right. Thanks for correcting it.
Original comment by jiayin...@gmail.com
on 17 May 2008 at 2:09
Here is a prototype implementation of btree update optimization.
Rather than optimizing all kinds of btree updates, the patch only optimizes the
journaling of dirty buffers generated during add_exception by recording the
add_exception change in the commit block instead of writing those dirty buffers
to
the journal. According to my measurement, dirty buffers generated during
add_exception are more than 90% of the total dirty buffers (not including bitmap
dirty buffers already saved by Daniel's bitmap optimization) during origin
writes. So
this approach should provide comparable performance as optimizing all kinds of
btree
updates, with the benefit of reduced complexity.
Basically, before entering add_exception update, we record the variables
required for
this change in ddsnap superblock. In commit_transaction, we copy the recorded
add_exception changes to commit block. All of the dirty buffers generated during
those add_exception changes are skipped during journal writes. Two fields are
added
to the ddsnap superblock: record_added_exceptions is the number of recorded
add_exception updates and added_exceptions is the array of recorded
add_exception
updates. An unsigned field is added to buffer structure to record the sequence
number
of add_exception changes. I also extended commit_block structure to cover two
kinds
of journaled change: the normal buffered journal change and the encoded
add_exception
change. During journal replay, these changes are scanned and recovered
accordingly.
I moved replay_journal backward because it needs to call add_exception_to_tree
now.
The optimization is turned off by default and can be enabled with the
--experimental|-X flag.
Patch is attached. Another attachment is the performance data measured on my
dell
workstation with nsnaps benchmark and 16k chunksize. In the graph, 'native' is
the
performance measured without experimental flag, 'experimental' is the
performance
measured with the current trunk code and with the experimental flag set (i.e.,
bitmap
optimization + deferred journal write optimization), and
'experimental+optimized_btree' is the performance with the patch applied (i.e.,
bitmap optimization + deferred journal write optimization + add_exception
optimization).
I got network outage problem with the uml cbtb tests. So it hasn't passed all
of the
uml tests. I will give it another try tomorrow.
Original comment by jiahotc...@gmail.com
on 11 Jul 2008 at 2:07
Attachments:
Original issue reported on code.google.com by
jiahotc...@gmail.com
on 9 Apr 2008 at 5:09