search

hawkw / sharded-slab

a lock-free concurrent slab (experimental)
MIT License
269 stars 17 forks source link

Unbounded memory growth with RESERVED_BITS #83

Closed lrowe closed 1 year ago

lrowe commented 1 year ago

I'm seeing unbounded memory growth when I set RESERVED_BITS to anything other than 0.

This is somewhat similar to #56 but the repro is less complex. Running on my Mac this is OOM killed after the 25th iteration.

Gist with repro and full outputs: https://gist.github.com/lrowe/56bc03b32e9d4bdabdd7d9f28f64af8d

main.rs

use sharded_slab::Config;
use sharded_slab::Slab;
use memory_stats::memory_stats;

struct CustomConfig;
impl Config for CustomConfig {
  const RESERVED_BITS: usize = 1;  // This is the cause.
}

fn main() {
  let slab = Slab::new_with_config::<CustomConfig>();
  for n in 0..10000 {
    let key = slab.insert(0).unwrap();
    slab.remove(key);
    let usage = memory_stats().unwrap();
    println!("n:{}\tkey:{}\trss:{}\tvs:{}", n, key, usage.physical_mem, usage.virtual_mem);
  }
}

cargo run output

n:0     key:0   rss:1523712     vs:418300739584
n:1     key:32  rss:1540096     vs:418300739584
n:2     key:96  rss:1540096     vs:418300739584
n:3     key:224 rss:1540096     vs:418300739584
n:4     key:480 rss:1556480     vs:418300739584
n:5     key:992 rss:1589248     vs:418300739584
n:6     key:2016        rss:1671168     vs:418434957312
n:7     key:4064        rss:1769472     vs:418434957312
n:8     key:8160        rss:1966080     vs:418434957312
n:9     key:16352       rss:2359296     vs:418434957312
n:10    key:32736       rss:3145728     vs:418434957312
n:11    key:65504       rss:4718592     vs:418434957312
n:12    key:131040      rss:7864320     vs:418434957312
n:13    key:262112      rss:14155776    vs:418434957312
n:14    key:524256      rss:26755072    vs:418447556608
n:15    key:1048544     rss:51920896    vs:418472722432
n:16    key:2097120     rss:102252544   vs:418523054080
n:17    key:4194272     rss:202915840   vs:418623717376
n:18    key:8388576     rss:404242432   vs:418825043968
n:19    key:16777184    rss:806895616   vs:419227697152
n:20    key:33554400    rss:1612201984  vs:420033003520
n:21    key:67108832    rss:3222814720  vs:421643616256
n:22    key:134217696   rss:6444040192  vs:424864841728
n:23    key:268435424   rss:6939295744  vs:431307292672
n:24    key:536870880   rss:4106715136  vs:444192194560
n:25    key:1073741792  rss:1419673600  vs:469961998336
zsh: killed     cargo run

vmmap summary before OOM killed

The virtual size grows absolutely huge, seemingly much bigger than that reported by the memory-stats crate.

==== Summary for process 81937
ReadOnly portion of Libraries: Total=809.2M resident=30.2M(4%) swapped_out_or_unallocated=779.1M(96%)
Writable regions: Total=96.6G written=58.2G(60%) resident=1.0G(1%) swapped_out=57.2G(59%) unallocated=38.3G(40%)

                                VIRTUAL RESIDENT    DIRTY  SWAPPED VOLATILE   NONVOL    EMPTY   REGION 
REGION TYPE                        SIZE     SIZE     SIZE     SIZE     SIZE     SIZE     SIZE    COUNT (non-coalesced) 
===========                     ======= ========    =====  ======= ========   ======    =====  ======= 
Kernel Alloc Once                   32K       0K       0K      16K       0K       0K       0K        1 
MALLOC guard page                   96K       0K       0K       0K       0K       0K       0K        5 
MALLOC metadata                    160K     160K     160K       0K       0K       0K       0K        8 
MALLOC_LARGE                      58.4G     1.0G     1.0G    57.2G       0K       0K       0K      470         see MALLOC ZONE table below
MALLOC_LARGE (reserved)           37.6G       0K       0K       0K       0K       0K       0K        1         see MALLOC ZONE table below
MALLOC_LARGE metadata               16K      16K      16K       0K       0K       0K       0K        1         see MALLOC ZONE table below
MALLOC_MEDIUM                     16.0M     144K     144K    11.8M       0K       0K       0K        2         see MALLOC ZONE table below
MALLOC_MEDIUM (empty)            112.0M       0K       0K       0K       0K       0K       0K       14         see MALLOC ZONE table below
MALLOC_NANO                      128.0M      96K      96K      96K       0K       0K       0K        1         see MALLOC ZONE table below
MALLOC_NANO (empty)              384.0M       0K       0K       0K       0K       0K       0K        3         see MALLOC ZONE table below
MALLOC_SMALL                      8192K      64K      64K      48K       0K       0K       0K        1         see MALLOC ZONE table below
MALLOC_TINY                       1024K      32K      32K       0K       0K       0K       0K        1         see MALLOC ZONE table below
Stack                             8160K      48K      48K       0K       0K       0K       0K        1 
Stack Guard                       56.0M       0K       0K       0K       0K       0K       0K        1 
VM_ALLOCATE                         32K       0K       0K       0K       0K       0K       0K        2 
VM_ALLOCATE (reserved)             128K       0K       0K       0K       0K       0K       0K        1         reserved VM address space (unallocated)
__AUTH                              46K      43K       0K     2608       0K       0K       0K       11 
__AUTH_CONST                        77K      77K       0K       0K       0K       0K       0K       40 
__DATA                             193K     105K      32K      87K       0K       0K       0K       39 
__DATA_CONST                       215K     167K       0K       0K       0K       0K       0K       42 
__DATA_DIRTY                        78K      36K      35K      43K       0K       0K       0K       22 
__LINKEDIT                       802.6M    24.8M       0K       0K       0K       0K       0K        2 
__OBJC_RO                         66.4M    45.1M       0K       0K       0K       0K       0K        1 
__OBJC_RW                         2012K    1456K       0K      28K       0K       0K       0K        1 
__TEXT                            6780K    5468K       0K       0K       0K       0K       0K       44 
dyld private memory                272K      32K      32K      16K       0K       0K       0K        2 
shared memory                       16K      16K      16K       0K       0K       0K       0K        1 
unused but dirty shlib __DATA       76K      29K      29K      47K       0K       0K       0K       24 
===========                     ======= ========    =====  ======= ========   ======    =====  ======= 
TOTAL                             97.5G     1.1G     1.0G    57.2G       0K       0K       0K      742 
TOTAL, minus reserved VM space    97.5G     1.1G     1.0G    57.2G       0K       0K       0K      742 

                                 VIRTUAL   RESIDENT      DIRTY    SWAPPED ALLOCATION      BYTES DIRTY+SWAP          REGION
MALLOC ZONE                         SIZE       SIZE       SIZE       SIZE      COUNT  ALLOCATED  FRAG SIZE  % FRAG   COUNT
===========                      =======  =========  =========  =========  =========  =========  =========  ======  ======
MallocHelperZone_0x10426c000       58.5G       1.0G       1.0G      57.2G         33      96.0G         0K      0%     489
DefaultMallocZone_0x1042a4000     128.0M        96K        96K        96K        225        10K       182K     95%       1
===========                      =======  =========  =========  =========  =========  =========  =========  ======  ======
TOTAL                              58.6G       1.0G       1.0G      57.2G        258      96.0G         0K      0%     490
hawkw commented 1 year ago

Interesting, thanks for the report!

loyd commented 1 year ago

Maybe related to https://github.com/hawkw/sharded-slab/pull/80. Although I don't see how yet

hawkw commented 1 year ago

I added a test for this in PR #83, and I can confirm that this issue was fixed by @loyd's changes from #80. I'm going to get a release out including the changes from #80 shortly, so this will be fixed in the next release.

hawkw commented 1 year ago

Okay, I've published v0.1.6 to crates.io, which should resolve this issue!