search

mfrw / compcache

Automatically exported from code.google.com/p/compcache
0 stars 0 forks source link

Comp-cache cannot handle >16G of swap space #15

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Load compcache on any machine with compcache_size_kbyts=20000000
2.
3.

What is the expected output? What do you see instead?
Expected - Cache of size 20G.
Happens - machine is stuck.

What version of the product are you using? On what operating system?
compcache 0.4.

Please provide any additional information below.
The problem happens due to a failure in allocating the tables with vmalloc.
16G of swap require (16G/pagesize)*16 (size of table entry) = 64M of 
memory.
Apparently, vmalloc cannot allocate more than 64M in a chunk (I believe 
this depends on OS configurations).

Obviously this is not a problem in small configuration, but on servers 
(yes, it's useful there too :)) it's an issue.

I have fixed this for the time being by using several compcache.table-s, 
each one relating to a different range (0-16, 16-32, etc.).

I can either commit the fix myself or let you fix it the way you choose.

BTW, I believe that when vmalloc fails (and it does return 0, it didn't 
hang), compcache shouldn't hang the machine, so might be nice to fix this 
as well.

Thanks!

Original issue reported on code.google.com by luna.sta...@gmail.com on 8 Oct 2008 at 12:39

GoogleCodeExporter commented 9 years ago 
Regarding the 64M limit on vmalloc, I believe it is coming from the fact 
vmalloc 
allocates page tables using kmalloc.
64M = 16k pages, and since the kmalloc limit is 128k, I am assuming each page 
info 
takes 8bytes ==> 16k pages = 128k.

So - that's probably the reason for the vmalloc limit.

Original comment by luna.sta...@gmail.com on 10 Oct 2008 at 6:11

GoogleCodeExporter commented 9 years ago 
I never thought anyone would even want to have this huge compcache sizes :)

I'm planning to replace this static array with radix trees. This should 
essentially
remove any size limits. However this can have some performance issues so lots of
testing will be required to make sure we are not hurting common case of smaller 
sizes
(< 100 MB).

Original comment by nitingupta910@gmail.com on 10 Oct 2008 at 6:50

GoogleCodeExporter commented 9 years ago 
Here's how I handled it:

#define CC_MAX_VMALLOC_IDX  ((1 << 22) - 1)

struct table* TABLE_ENTRY(unsigned long idx)
{
  if (idx < CC_MAX_VMALLOC_IDX) return &compcache.table[idx];
  return &compcache.table2[idx - CC_MAX_VMALLOC_IDX];
}

and every compcache.table[page_no]. usage replaced with TABLE_ENTRY(page_no)->

I think it might be a good implementation here as well, since it bears no 
runtime 
damage.
You can even lower the size so each table is much smaller (~512k?) and then 
allocate 
multiple tables (array of arrays).

Original comment by luna.sta...@gmail.com on 10 Oct 2008 at 3:51

GoogleCodeExporter commented 9 years ago 
I'm now working on another solution. There is now experimental support to create
multiple ramzswap devices. So, instead of having a single mammoth 16G ramzswap, 
you
could use multiple ramzswap devices -- this will also help with scalability and 
each
device uses separate xvmalloc pools, compression buffers etc.

Original comment by nitingupta910@gmail.com on 18 Jun 2009 at 3:52

GoogleCodeExporter commented 9 years ago 
Just an fyi: :"I never thought anyone would even want to have this huge 
compcache"

I am thinking about this for machine which have lots of ram, say 256/512GB  but 
where
the application may want 1TB.

I would also like it on my home mips nas box :) with 256Mb of ram.

Original comment by zz9p...@gmail.com on 22 Jun 2009 at 8:18

GoogleCodeExporter commented 9 years ago 
compcache-0.6 (currently 0.6-pre2) implements multiple ramzswap devices, as 
mentioned
in comment #4. So, following the theme of 'divide and conquer', you can now 
create
multiple ramzswap devices with each having a different backing swap 
file/partition
instead of a single gigantic ramzswap device.

Original comment by nitingupta910@gmail.com on 20 Jul 2009 at 5:39

GoogleCodeExporter commented 9 years ago 
As mentioned in previous comment, with multiple ramzswap devices, practically 
there
in no limit to total size of rzs devices we can have. So, closing this issue.

Original comment by nitingupta910@gmail.com on 30 Jul 2009 at 5:58