search

liuis / leveldb

Automatically exported from code.google.com/p/leveldb
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

Using mmap for random reads for TableCache files uses a lot of memory #116

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Fix db/db_bench.cc to respect open_files -- 
http://code.google.com/p/leveldb/issues/detail?id=115 -- and then run a test 
with a database that has 10,000 files and --cache_size=1G. RSS for the db_bench 
process will be greater than 20G because each file used by an entry in the 
TableCache uses 2MB.

Given that LevelDB uses small files by default, big LevelDB databases will have 
many files and this is too much memory to use for the table cache. Or is the 
use of small files by db_bench not the right way to run big LevelDB databases?

Original issue reported on code.google.com by mdcal...@gmail.com on 14 Sep 2012 at 11:39

GoogleCodeExporter commented 9 years ago 
I ran into the mmap issue causing the app to crash on iOS. Any ideas how to 
reduce memory usage on iOS?I wish the documentation was clearer on tuning 
leveldb...

Original comment by alexkarp...@gmail.com on 15 Sep 2012 at 11:26

GoogleCodeExporter commented 9 years ago 
More details on the performance impact from this are in this thread:
https://groups.google.com/forum/?fromgroups=#!topic/leveldb/mkKRKA4XGb0

The basic problem is that a LevelDB process will try to get as much memory as 
possible as reported by RSS to cache the database in its address space. I don't 
that is good behavior either for an embedded or a server database.

Original comment by mdcal...@gmail.com on 18 Sep 2012 at 4:02

GoogleCodeExporter commented 9 years ago 
Fixed in 1.6.0

Original comment by san...@google.com on 12 Oct 2012 at 8:02