search

harsh97 / compcache

Automatically exported from code.google.com/p/compcache
1 stars 0 forks source link

Scalability improvements #24

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Load compcache
2. Run several intensive processes on the machine
3.

What is the expected output? What do you see instead?

All processes should be on 100% CPU utilization (my machine has 4 cores).

What version of the product are you using? On what operating system?

0.5.1 on 2.6.16.

Please provide any additional information below.

Currently there's a mutex protecting compression since there's only one 
compression buffer.
It would be better to keep several compression buffers and have them 
working in parallel. I did this fix (dirty :)) and noticed that it's even 
possible for 10 different requests to be sent to comp-cache 
simultaneously. Enabling more writes to happen in parallel by keeping 
several buffers improves performance.

Original issue reported on code.google.com by luna.sta...@gmail.com on 5 Feb 2009 at 9:31

GoogleCodeExporter commented 9 years ago 
Another bottleneck is memory allocator - per-pool spinlock. So, only one 
alloc/free
can happen at a time. Bad. Someday I will improve these ...

Original comment by nitingupta910@gmail.com on 6 Feb 2009 at 12:22

GoogleCodeExporter commented 9 years ago 
I will be happy to help with this. Would you like me to start by allowing 
multiple 
compression buffers? If so, I'll sketch a simple algorithm and mail it over for 
your 
review

Original comment by luna.sta...@gmail.com on 6 Feb 2009 at 1:45

GoogleCodeExporter commented 9 years ago 
> I will be happy to help with this. Would you like me to start by allowing 
multiple 
> compression buffers? If so, I'll sketch a simple algorithm and mail it over 
for your 
> review

This would be nice!

Original comment by nitingupta910@gmail.com on 6 Feb 2009 at 2:29

GoogleCodeExporter commented 9 years ago 
Original bug summary seems to suggest that compcache 'crashes' on multi-core 
systems.

Original comment by nitingupta910@gmail.com on 2 Apr 2009 at 9:25

GoogleCodeExporter commented 9 years ago 
compcache-0.6 (currently 0.6-pre2) implements multiple ramzswap devices. I 
think by
creating multiple ramzswap devices, you can partly address this scalability 
issue
since each of these devices has independent buffers, xvmalloc pool etc.

Original comment by nitingupta910@gmail.com on 20 Jul 2009 at 5:42

GoogleCodeExporter commented 9 years ago 
See: http://code.google.com/p/compcache/wiki/Scalability
HTH.

Original comment by nitingupta910@gmail.com on 5 Jan 2010 at 5:27

GoogleCodeExporter commented 9 years ago 
With support for creating multiple devices, scalability shouldn't be an issue. 
Still, there are many things to improve in this area but nothing very critical 
now, I think.

Original comment by nitingupta910@gmail.com on 4 Jul 2010 at 2:39