search

mfrw / compcache

Automatically exported from code.google.com/p/compcache
0 stars 0 forks source link

Allow specifying numdevices during kernel config and as boot parameter #87

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1.

echo $((300*1024*1024)) > /sys/block/zram0/disksize
mkfs.ext4 /dev/zram0
mount /dev/zram0 /test

2.
put any file with good size (zip or anything  with size under 300MB
in my case a file of 273MB

cat /sys/block/zram1/mem_used_total :209244160

3.remove the file 
rm file

cat /sys/block/zram1/mem_used_total :209244160

i using a linux 2.6.38 zram module

the point is that zram device only grow size overtime but unlike ramfs when the 
device is freed (aka there is nothing in the device) , in zram the memory is 
not freed !

this is not optimal since you need manualy reset the device and redo the fs 
after every use

Original issue reported on code.google.com by projetiu...@gmail.com on 27 Apr 2011 at 6:00

GoogleCodeExporter commented 9 years ago 
I just committed block "discard" support which lets filesystems know that the 
device would like to know when a block/sector is freed. We use this to free 
corresponding compressed pages.

Following shows stats of zram0 after it was filled with files from /usr/lib64 
and subsequently all these files were deleted, so compressed data size 
(compr_data_size) is now only ~86K, but wait... what's that HUGE mem_used_total 
(~4.25 MB)??? -- that's the memory allocator overhead which suffers badly due 
to fragmentation when lots of chunks are freed and much is reallocated.  I plan 
to start working on a completely new allocator soon which addresses these 
issues and many more...

disksize:          268435456
num_reads:               175
num_writes:             9703
invalid_io:                0
discard:                  10
notify_free:               0
zero_pages:             6025
orig_data_size:       618496
compr_data_size:       85962
mem_used_total:       425984
avg_compr_ratio:          13 %
mem_overhead:            395 %

Original comment by nitingupta910@gmail.com on 28 Apr 2011 at 2:14

GoogleCodeExporter commented 9 years ago 
One more thing, ext4 needs discard to be explicitly enabled during mount, so 
you need to do:
mkfs.ext4 /dev/zram0
mount /dev/zram0 /test -o discard

and similarly when using as swap:
mkswap /dev/zram0
swapon /dev/zram0 --discard

Original comment by nitingupta910@gmail.com on 28 Apr 2011 at 1:11

GoogleCodeExporter commented 9 years ago 
hello ,

thanks for the discard support, it's really great!
after one week of compilation and testing , this option allow me now to have 
several compilation without running out of memory.

by the way i t would be great , if this option is commited to zram in kernel 
because it increase zram usefulness for ext4 and swap.

moreover it would great to allow in kernel the ability of choose the number of 
devices by default.

since zcache need zram to be compiled not in module form, this options would be 
great in order to have more than one device(for example have one device for 
swap and/or compilation tmp)

bye

Original comment by projetiu...@gmail.com on 6 May 2011 at 8:40

GoogleCodeExporter commented 9 years ago 
> by the way i t would be great , if this option is commited to zram in kernel 
because it increase zram usefulness for ext4 and swap.

Yes, I will be sending a patch for inclusion in the mainline soon. BTW, this 
discard support is not of much use for swap case since kernel already has a 
specialized callback for this swap case which allows us to free memory as soon 
as a swap block is freed 
(http://driverdev.linuxdriverproject.org/pipermail/devel/2010-May/005502.html). 
Discard is more "lazy" callback approach used by filesystems.

> moreover it would great to allow in kernel the ability of choose the number 
of devices by default.

Ok, I will be adding a zram numdevices are boot parameter and are kernel config 
option to set it at compile time.

---

Changing Issue summary to reflect what needs to be done now

Original comment by nitingupta910@gmail.com on 6 May 2011 at 2:26

GoogleCodeExporter commented 9 years ago 
hello ,

i would like to know if you can  send a patch for inclusion in the mainline 
(the 3.0 would be good) for the discard option that don't exist on mainline 
kernel.

moreover , adding a zram numdevices as boot parameter and as kernel config 
option to set it at compile time as patch for th mainline would be very useful 
for people(like me) that use more than 1 device by default

thanks

Original comment by projetiu...@gmail.com on 20 Jun 2011 at 12:21

GoogleCodeExporter commented 9 years ago 
hello , 

Can you provide a patch for the curent kernel(2.6.39 or 3.0rc4) for:
-the discard option 
-zram numdevices as boot parameter & as kernel config option
on a same file.

thanks

Original comment by projetiu...@gmail.com on 27 Jun 2011 at 3:27

GoogleCodeExporter commented 9 years ago 
I have uploaded zram patch (applies to 3.0-rc3) with numdevices as kernel 
config option and the discard support.  I will be sending this for mainline 
inclusion soon.

Original comment by nitingupta910@gmail.com on 29 Jun 2011 at 6:34