search

pmem / CacheLib

Pluggable in-process caching engine to build and scale high performance services
https://www.cachelib.org
Apache License 2.0
5 stars 13 forks source link

Add memory usage statistics for slabs and allocation classes #91

Closed igchor closed 2 years ago

igchor commented 2 years ago

Printing those stats can be enabled by passing -report_memory_usage_stats to cachebench

Sample output:

tid 0 free slabs : 0.00%
tid 1 free slabs : 0.00%
tid 0 pid 0 cid   0    64.00B memorySize:     0.00B
tid 0 pid 0 cid   1    96.00B memorySize:    36.00MB
tid 0 pid 0 cid   2   144.00B memorySize:   124.00MB
tid 0 pid 0 cid   3   216.00B memorySize:   276.00MB
tid 0 pid 0 cid   4   328.00B memorySize:   744.00MB
tid 0 pid 0 cid   5   496.00B memorySize:   992.00MB
tid 0 pid 0 cid   6   744.00B memorySize:   508.00MB
tid 0 pid 0 cid   7     1.09KB memorySize:   416.00MB
tid 0 pid 0 cid   8     1.64KB memorySize:   212.00MB
tid 0 pid 0 cid   9     2.46KB memorySize:   132.00MB
tid 0 pid 0 cid  10     3.70KB memorySize:   152.00MB
tid 0 pid 0 cid  11     5.55KB memorySize:   136.00MB
tid 0 pid 0 cid  12     8.32KB memorySize:    36.00MB
tid 0 pid 0 cid  13    12.48KB memorySize:    20.00MB
tid 0 pid 0 cid  14    18.78KB memorySize:    20.00MB
tid 0 pid 0 cid  15    28.24KB memorySize:    32.00MB
tid 0 pid 0 cid  16    42.66KB memorySize:   200.00MB
tid 0 pid 0 cid  17    64.00KB memorySize:    36.00MB
tid 0 pid 0 cid  18    97.52KB memorySize:    12.00MB
tid 0 pid 0 cid  19   151.70KB memorySize:     4.00MB
tid 0 pid 0 cid  20   227.55KB memorySize:     4.00MB
tid 0 pid 0 cid  21   372.36KB memorySize:     0.00B
tid 0 pid 0 cid  22   585.14KB memorySize:     0.00B
tid 0 pid 0 cid  23     1.00MB memorySize:     0.00B
tid 1 pid 0 cid   0    64.00B memorySize:     0.00B
tid 1 pid 0 cid   1    96.00B memorySize:    40.00MB
tid 1 pid 0 cid   2   144.00B memorySize:   124.00MB
tid 1 pid 0 cid   3   216.00B memorySize:   272.00MB
tid 1 pid 0 cid   4   328.00B memorySize:   760.00MB
tid 1 pid 0 cid   5   496.00B memorySize:   984.00MB
tid 1 pid 0 cid   6   744.00B memorySize:   508.00MB
tid 1 pid 0 cid   7     1.09KB memorySize:   424.00MB
tid 1 pid 0 cid   8     1.64KB memorySize:   208.00MB
tid 1 pid 0 cid   9     2.46KB memorySize:   136.00MB
tid 1 pid 0 cid  10     3.70KB memorySize:   148.00MB
tid 1 pid 0 cid  11     5.55KB memorySize:   132.00MB
tid 1 pid 0 cid  12     8.32KB memorySize:    32.00MB
tid 1 pid 0 cid  13    12.48KB memorySize:    16.00MB
tid 1 pid 0 cid  14    18.78KB memorySize:    20.00MB
tid 1 pid 0 cid  15    28.24KB memorySize:    32.00MB
tid 1 pid 0 cid  16    42.66KB memorySize:   208.00MB
tid 1 pid 0 cid  17    64.00KB memorySize:    32.00MB
tid 1 pid 0 cid  18    97.52KB memorySize:    12.00MB
tid 1 pid 0 cid  19   151.70KB memorySize:     4.00MB
tid 1 pid 0 cid  20   227.55KB memorySize:     0.00B
tid 1 pid 0 cid  21   372.36KB memorySize:     0.00B
tid 1 pid 0 cid  22   585.14KB memorySize:     0.00B
tid 1 pid 0 cid  23     1.00MB memorySize:     0.00B
tid 0 pid 0 cid   0    64.00B free: 100.00%
tid 0 pid 0 cid   1    96.00B free: 0.00%
tid 0 pid 0 cid   2   144.00B free: 0.00%
tid 0 pid 0 cid   3   216.00B free: 0.00%
tid 0 pid 0 cid   4   328.00B free: 0.00%
tid 0 pid 0 cid   5   496.00B free: 0.00%
tid 0 pid 0 cid   6   744.00B free: 0.01%
tid 0 pid 0 cid   7     1.09KB free: 0.02%
tid 0 pid 0 cid   8     1.64KB free: 0.02%
tid 0 pid 0 cid   9     2.46KB free: 0.02%
tid 0 pid 0 cid  10     3.70KB free: 0.04%
tid 0 pid 0 cid  11     5.55KB free: 0.06%
tid 0 pid 0 cid  12     8.32KB free: 0.06%
tid 0 pid 0 cid  13    12.48KB free: 0.03%
tid 0 pid 0 cid  14    18.78KB free: 0.04%
tid 0 pid 0 cid  15    28.24KB free: 0.02%
tid 0 pid 0 cid  16    42.66KB free: 0.01%
tid 0 pid 0 cid  17    64.00KB free: 0.00%
tid 0 pid 0 cid  18    97.52KB free: 0.00%
tid 0 pid 0 cid  19   151.70KB free: 0.00%
tid 0 pid 0 cid  20   227.55KB free: 61.11%
tid 0 pid 0 cid  21   372.36KB free: 100.00%
tid 0 pid 0 cid  22   585.14KB free: 100.00%
tid 0 pid 0 cid  23     1.00MB free: 100.00%
tid 1 pid 0 cid   0    64.00B free: 100.00%
tid 1 pid 0 cid   1    96.00B free: 0.00%
tid 1 pid 0 cid   2   144.00B free: 0.00%
tid 1 pid 0 cid   3   216.00B free: 0.00%
tid 1 pid 0 cid   4   328.00B free: 0.00%
tid 1 pid 0 cid   5   496.00B free: 0.00%
tid 1 pid 0 cid   6   744.00B free: 0.01%
tid 1 pid 0 cid   7     1.09KB free: 0.02%
tid 1 pid 0 cid   8     1.64KB free: 0.02%
tid 1 pid 0 cid   9     2.46KB free: 0.02%
tid 1 pid 0 cid  10     3.70KB free: 0.04%
tid 1 pid 0 cid  11     5.55KB free: 0.06%
tid 1 pid 0 cid  12     8.32KB free: 0.06%
tid 1 pid 0 cid  13    12.48KB free: 0.03%
tid 1 pid 0 cid  14    18.78KB free: 0.04%
tid 1 pid 0 cid  15    28.24KB free: 0.02%
tid 1 pid 0 cid  16    42.66KB free: 0.01%
tid 1 pid 0 cid  17    64.00KB free: 0.00%
tid 1 pid 0 cid  18    97.52KB free: 0.00%
tid 1 pid 0 cid  19   151.70KB free: 0.00%
tid 1 pid 0 cid  20   227.55KB free: 100.00%
tid 1 pid 0 cid  21   372.36KB free: 100.00%
tid 1 pid 0 cid  22   585.14KB free: 100.00%
tid 1 pid 0 cid  23     1.00MB free: 100.00%

This change is Reviewable

raema commented 2 years ago

Hi @igchor, most of the metrics have the name and value separated by a single colon like: 'Hit Ratio : 92.07%' That metric appears in prometheus named 'hit_ratio'

For the line below, I'm thinking how we would parse this output into 'name, value' for prometheus tid : 0, pid : 0, cid : 0, alloc size : 64.00B, memory size : 0.00B, free : 100.00%

If we want to follow the same format as other metrics maybe we could use this format: tid0 pid0 cid0 64.00KB size : 0.00B tid0 pid0 cid0 64.00KB free : 100.00%

Does it make sense to change the stats output to use this format or something like this? Our existing scripts will automatically collect new metrics output in this format.

guptask commented 2 years ago

cachelib/allocator/CacheAllocator-inl.h line 2510 at r1 (raw file):


template <typename CacheTrait>
double CacheAllocator<CacheTrait>::slabsFreePercentage(TierId tid) const

should this name have approx term in it as well ?

guptask commented 2 years ago

cachelib/allocator/memory/AllocationClass.cpp line 54 at r1 (raw file):

      slabAlloc_(s),
      freedAllocations_{slabAlloc_.createSingleTierPtrCompressor<FreeAlloc>()} {
  curAllocatedSlabs_ = allocatedSlabs_.size();

Do you want to use memory_order_seq_cst order here for atomic store ? Otherwise store() is needed if you want to use something like memory_order_release order.

guptask commented 2 years ago

cachelib/allocator/memory/AllocationClass.cpp line 126 at r1 (raw file):

  }

  curAllocatedSlabs_ = allocatedSlabs_.size();

same as above comment about memory order if using assignment operator instead of atomic store().

guptask commented 2 years ago

cachelib/allocator/memory/AllocationClass.cpp line 741 at r1 (raw file):


double AllocationClass::approxFreePercentage() const {
  if (getNumSlabs() == 0)

Cosmetic review : I suggest using curly braces even if there is one line after if statement.

guptask commented 2 years ago

cachelib/allocator/tests/CacheBaseTest.cpp line 36 at r1 (raw file):

  const MemoryPool& getPool(PoolId) const override { return memoryPool_; }
  PoolStats getPoolStats(PoolId) const override { return PoolStats(); }
  AllocationClassBaseStat getAllocationClassStats(TierId tid, PoolId, ClassId) const {return AllocationClassBaseStat();};

am not sure this line will pass the clang-format check. the character count is high.

raema commented 2 years ago

@raema Ok, I changed it to how you suggested. Please let me now if this is OK.

This output looks good. Thanks!