stec-inc / EnhanceIO

EnhanceIO Open Source for Linux
Other
420 stars 176 forks source link

Why are sets protected by rw-lock? #118

Open sebuns opened 8 years ago

sebuns commented 8 years ago

I'm PhD student and doing research based on EnhanceIO.

I heard that EIO is forked from flash-cache. So, I compare two SSD-caches. For evaluation, I ran same benchmark (fio) on EIO and Facebook's flash-cache.

With EXT4 (ordered journaling), flash-cache shows much better performance than EIO. (measured bandwidths are 40,891 KiB/s (flash-cache) and 23,988 KiB/s (EIO) each)

I think rw-lock of each cache_sets makes this performance gap. Since journaled write is sequential write, it dirties several specific cache_sets easily. It means that, they are much likely to be cleaned by write-back thread than other cache_sets. More seriously, because it is synchronous write, if journaled write is blocked, it hurts application's performance significantly.

My question is why EIO protects each set by rw-lock.

In terms of memory size of cache metadata, EIO got a significant benefit. However, I think, because EIO takes course-grained lock for protecting cache_set and its on-flash metadata, it can hurt its performance.

Is this just for memory size issue? Or, did I miss some important details in design or implementation?

Thanks.

-Sung-hun