search

facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
28.67k stars 6.33k forks source link

Persistent Cache tests fail on Windows #4177

Open yuslepukhin opened 6 years ago

yuslepukhin commented 6 years ago

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://www.facebook.com/groups/rocksdb.dev

Expected behavior

tests pass

Actual behavior

The following test cases fail in a similar fashion. This is v5.14.2 tag build.

WARNING: persistent_cache_test::PersistentCacheDBTest.TieredCacheTest State: Completed WARNING: Note: Google Test filter = PersistentCacheDBTest.TieredCacheTest WARNING: [==========] Running 1 test from 1 test case. WARNING: [----------] Global test environment set-up. WARNING: [----------] 1 test from PersistentCacheDBTest WARNING: [ RUN ] PersistentCacheDBTest.TieredCacheTest WARNING: d:\dev\rocksdb\repo.merge\utilities\persistent_cache\persistent_cache_test.cc(423): error: Expected: (page_hit) > (0), actual: 0 vs 0

WARNING: persistent_cache_test::PersistentCacheDBTest.BlockCacheTest State: Completed WARNING: Note: Google Test filter = PersistentCacheDBTest.BlockCacheTest WARNING: [==========] Running 1 test from 1 test case. WARNING: [----------] Global test environment set-up. WARNING: [----------] 1 test from PersistentCacheDBTest WARNING: [ RUN ] PersistentCacheDBTest.BlockCacheTest WARNING: d:\dev\rocksdb\repo.merge\utilities\persistent_cache\persistent_cache_test.cc(423): error: Expected: (page_hit) > (0), actual: 0 vs 0

WARNING: persistent_cache_test::PersistentCacheDBTest.BlockCacheTest State: Completed WARNING: Note: Google Test filter = PersistentCacheDBTest.BlockCacheTest WARNING: [==========] Running 1 test from 1 test case. WARNING: [----------] Global test environment set-up. WARNING: [----------] 1 test from PersistentCacheDBTest WARNING: [ RUN ] PersistentCacheDBTest.BlockCacheTest WARNING: d:\dev\rocksdb\repo.merge\utilities\persistent_cache\persistent_cache_test.cc(423): error: Expected: (page_hit) > (0), actual: 0 vs 0

Steps to reproduce the behavior

Run the tests

siying commented 6 years ago

Do you use persistent cache? We have been discussing to deprecate it.

yuslepukhin commented 6 years ago

@siying I am not using it but I am curious to see what is broken and if that is related somehow to the other problems. None of the inserted keys hit it. I mean why it would stop working?

maysamyabandeh commented 6 years ago

@yuslepukhin assigning it to you as you are our windows expert.

siying commented 6 years ago

@maysamyabandeh assigning the issue to who opened it?

Ye-Tian-Zero commented 6 years ago

@siying Why do you want to deprecate persistent cache?

ajkr commented 6 years ago

@skilxnTL we don't use it. Do you?

Ye-Tian-Zero commented 6 years ago

@ajkr We try to use it, but there are some problems. Each k-v element will cost about 200 bytes in block-cache-tier's meta data. If we use a 800G ssd, and 4kB block size for sst-files, block-cache-tiers's meta data will cost 800 1024 1024(kB) / 4kB * 200 bytes = 40G memory.

Have you test this?

siying commented 6 years ago

@skilxnTL yes we tested it and we are aware that the memory consumption is a little bit high. However, we never plan a 800GB SSD as a cache. We are more thinking of 100+GB persistent cache. Memory consumption is one another reason we want to deprecate it is that we want to implement something that is much more memory efficient.

maysamyabandeh commented 5 years ago

@siying Is persistent cache still going to be deprecated?

siying commented 5 years ago

@maysamyabandeh well, though it's not in active development and we don't suggest everyone use it, it's still good for some experiments, so it may not be a good idea to remove the code, unless we have a good replacement for experiments.