search

akiradeveloper / dm-writeboost

Log-structured Caching for Linux
GNU General Public License v2.0
120 stars 19 forks source link

How can dm-writeboost know about disk load performace for writeback-threshold? #204

Closed nhocchi007 closed 5 years ago

nhocchi007 commented 5 years ago

You said: "Writeback can be suppressed when the load of backing device is higher than $writeback_threshold", as my guess, we talking about writeback-cache procedure and performace load of backing device right? Suppose i have 10 backing device running at raid 0, or raid 1. How can dm-writeboost calculate these backing load performance for writeback-threshold? And how can i calculate time when dm-writeboost write from cache device to backing device? Thank you for your attention

akiradeveloper commented 5 years ago

@nhocchi007 In writeback_modulator_proc you can see writeboost exploits part_stat_read and it roughly means writeboost tries to know how much time backing store has dedicated to IOs in the most recent one second.

To know how much time writeboost has dedicated to writeback, you may use sysstat. Basically, these information is not important for most of the users and writeboost will not provide such stat in kernel.

nhocchi007 commented 5 years ago

I tried to build a system with high performance on writing data only use least amount of ssd and mostly hdd, i have to assure same amount of data must be writing in 1 day, and writing continuosly. But currently, i can only use practical test on disk performace when using 1 cache device and 1 backing device, and assume linear algebra will work with larger amount of devices. Of course, i have consider about elements that will effect disk performance(RAID, real disk performace on specific type of data...). So i assume my backing devices will always working as highest load, right? Is the number writeback threshold 70 is meaningless now?

akiradeveloper commented 5 years ago

I could not understand what you are trying to build but for the question itself it clear.

Except test/experimental situation, I recommend to set 70. The 70 is not heuristic but comes from mathematics (of queuing theory). It is theoretically the best number for the ave. throughput at least.

nhocchi007 commented 5 years ago

So in this situation: My cache device (ssd) is full, what will happen? And what you means by "Writeback can be suppressed " ? Data lost?

akiradeveloper commented 5 years ago

It is just suppressing writeback and there is no room in the caching device to write new data then all the succeeding requests will just wait in queue.

nhocchi007 commented 5 years ago

Can we control when the cached device write to backing device? and when do caching device clear itself ?

akiradeveloper commented 5 years ago

I don't think we practically are in such situation but trying to writeback as long as the backing device is not busy is the best strategy. In my tests (https://github.com/akiradeveloper/writeboost-test-suite) I sometimes control it but it is not the normal usage of this module but to check the logic in sequence. I don't understand the second question. what is clear? triggering re-formatting can be kicked by zero-ing the first sector of the caching device however, which is written in the document please read it.

nhocchi007 commented 5 years ago

Thank you so much! i will