facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
28.63k stars 6.33k forks source link

Add `strict_capacity_limit` block cache option to stress/crash tests #7559

Open ajkr opened 4 years ago

ajkr commented 4 years ago

As we learned recently with bugs in index partition pinning, there may be bugs when a subset of reads go through block cache. One way that we might expose such bugs is by adding strict_capacity_limit to stress/crash tests to make block insertions fail at essentially arbitrary points. This would also give us confidence that our handling of block cache insertion failures is correct.

This issue requires some investigation. It's unknown whether flush/compaction can succeed in case of a full block cache with strict_capacity_limit == true. It's also unclear how to retry a user operation after it fails due to full block cache (maybe ReadOptions::fill_cache == false?).

mrambacher commented 4 years ago

Going through the ASSERT_STATUS_CHECKED checks, there are many places where the code simply ignores errors from inserting into the Cache. This problem is along the same lines as you mentioned, but perhaps if there was some better way of handling or notifying those errors, something could be done in a more standard/common way...