My implementation: Compaction IO speed limitation

[GoogleCodeExporter]

Hi, everyone:

I created an issue days 
ago(https://code.google.com/p/leveldb/issues/detail?id=168&sort=-id), to 
require the feature of compaction IO speed limitaion. But no reponse.

As we know, the leveldb has extremly BAD performance during compaction, because 
it eats up all the disk IO capacity. So a compaction IO speed limitation 
feature is very important.

I wrote my own. Here is some of the codes:

diff --git a/deps/leveldb-1.9.0/db/db_impl.cc b/deps/leveldb-1.9.0/db/db_impl.cc
index c9de169..56915f4 100644
--- a/deps/leveldb-1.9.0/db/db_impl.cc
+++ b/deps/leveldb-1.9.0/db/db_impl.cc
@@ -805,6 +805,22 @@ Status DBImpl::FinishCompactionOutputFile(CompactionState* 
compact,
           (unsigned long long) output_number,
           (unsigned long long) current_entries,
           (unsigned long long) current_bytes);
+
+               // Writing current_bytes to disk is considered no expense(cost 
no time),
+               // so we calculate how many IOs will match the compaction speed,
+               // then sleep 1s/IOs
+               // Added by me@ideawu.com
+               int mbs = current_bytes/1024/1024;
+               if(options_.compaction_speed > 0 && mbs > 1){
+                       int count = options_.compaction_speed/mbs;
+                       if(count < 1){
+                               count = 1;
+                       }
+                       int pause = 1000 * 1000 / count;
+                       Log(options_.info_log, "compaction_speed: %d MB, pause: 
%d us",
+                               options_.compaction_speed, pause);
+                       env_->SleepForMicroseconds(pause);
+               }
     }
   }
   return s;

This is a very primative approach, I am expecting there is an official 
implementation.

The full codes is here: 
https://github.com/ideawu/ssdb/tree/master/deps/leveldb-1.9.0

---

Original issue reported on code.google.com by wuzuy...@gmail.com on 15 Jun 2013 at 5:28