compcache 0.4 crashes on ARM

[GoogleCodeExporter]

What steps will reproduce the problem?
1. use compcache by swapon /dev/compcache0 -p 1or swapon /dev/compcache -p 1
2.
3.

What is the expected output? What do you see instead?
frozen screen

What version of the product are you using? On what operating system?
latest. ubuntu 7.10 , 8.04 alpha 5, debian-eabi 2.6.24, 2.6.23, 2.6.21

Please provide any additional information below.
 this project is not stable, it will lock up any linux machine, SMP or not,
embedded or desktop.

Original issue reported on code.google.com by marblema...@gmail.com on 3 Mar 2008 at 8:01

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

tried on Athlon 64 3500+ Socket 939, pxa270, pxa255, all frozen after any load 
is
invoked... ive NEVER seen linux freeze befor! am i not supposed to use real swap
partitions/files while using compcache? because i have been. im going to try 
without
using real swap now. I have noticed a big performance boost in my pxa270 with 
64mb
SDRAM! instead of using 10MB of swap im only using 4kb! using compcache alone 
seems
stable, i think using real swap with compcache is whats making it freeze. Why 
doesnt
it mention this in the wiki?

Original comment by marblema...@gmail.com on 3 Mar 2008 at 9:52

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

just got it to repeat the same issue without any swap active. Using compcache 
6mb
alone got a pxa270 using 64mb sdram non smp kernel 2.6.24 froze while trying to
install xxdiff armel .deb :( 

this compcache 0.1 doesnt seem stable, ill try compcache 0.2 

Original comment by marblema...@gmail.com on 4 Mar 2008 at 1:06

[GoogleCodeExporter]

Please let me know if you get these crashes with compcache 0.2.

Anyway all testing I have done is on x86. I again tested 0.2 any my x86 box 
(Fedora VM):
RAM: 256M
swap(hdisk): 1G
swap(compcache): 64M

With above config I never had any crashes. I filled entire compcache so that it
starts swapping to hdisk also and it worked without any issues.

Original comment by nitingupta910@gmail.com on 4 Mar 2008 at 8:13

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

what kernel are you using? I cannot get compcache.ko to compile on 2.6.24 :( 
I am also testing on x86 first, then if i ever get a stable build on x86, ill 
use it
on my ARMV5T PXA255 and PXA270 i have an EM-X270 and Zaurus SL-C1000, and Ipaq 
h5555,
all running linux 2.6.24 debian-eabi full. Its like having ubuntu in your 
pocket,
99.9% of all the packages work but theres only 64mb ram or 128mb on the ipaq, it
worked amazing when it was not frozen, the performance is amazing. I only wish 
it
wouldent freeze....

Original comment by marblema...@gmail.com on 5 Mar 2008 at 8:24

[GoogleCodeExporter]

I am using 2.6.23 and 2.6.25-rc3. Can you please post what compile errors you 
are
getting? Maybe you don't have kernel source present on your system?

Original comment by nitingupta910@gmail.com on 5 Mar 2008 at 8:29

[GoogleCodeExporter]

wasent this project focused twards swapless embedded systems?
I just found out all i had to do was add #include <linux/vmalloc.h> to 
compcache.c 
 ( ^  ^

so ill try it asap, and report my results with compcache 0.2, my friend who has
already compiled the modules says it still freezes up :/

i only wish this would work without freezing, the performance increase i got was
something really amazing...

Original comment by marblema...@gmail.com on 5 Mar 2008 at 8:45

[GoogleCodeExporter]

actually ive been compileing kernel sources for a while now and symlinking them 
to
/lib/modules/Kernelnamehere/build
 ( ^  ^

everything is building fine, its just not running fine.

Original comment by marblema...@gmail.com on 5 Mar 2008 at 8:46

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

2.6.24 is too fast for me to leave... i wont ever go back to 2.6.23...

Original comment by marblema...@gmail.com on 5 Mar 2008 at 8:47

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Thanks for the info. I don't know how it compiles just fine without including
vmalloc.h on my system - I have now added it to compcache.c (commited to svn).

Can you provide kernel logs from the system where it is crashing?

Original comment by nitingupta910@gmail.com on 5 Mar 2008 at 9:01

[GoogleCodeExporter]

Without alteast the logs I have no idea why its crashing on your system.
Surprisingly, it never crashes for me (see test config in comment #8).

Original comment by nitingupta910@gmail.com on 5 Mar 2008 at 9:03

[GoogleCodeExporter]

OK i compiled and ran the use script in x86 single core ubuntu 8.04 alpha 5 
running
2.6.24-11.29.....

Frozen.

This is not good.
:(

it takes a while for it to crash tho

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:08

[GoogleCodeExporter]

yonggun@gmail.com was the one who told me to put
#include <linux/vmalloc.h> into compcache.c 

thats how i got it to compile on my pocket pc. It works somehow without it on 
my x86
tho. Anyway im using gcc 4.2.3 and kernel 2.6.24 , i hope that isint a problem
because i keep getting frozen after like a few hours or a half hour of use...

how do i get to these logs? im willing to send any information i have to deal 
with
this issue..

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:11

[GoogleCodeExporter]

im also using stress, crashme and john -test to test stability

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:12

[GoogleCodeExporter]

its crashing ubuntu 7.10 single core, 8.04 alpha 5, and debian-eabi lenny/sid 
all
2.6.24 ofcourse. As soon as i find out where the logs are located on my system 
i will
paste them into a comment here

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:16

[GoogleCodeExporter]

and is it ok to use this on a dual-core system?

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:18

[GoogleCodeExporter]

> how do i get to these logs? im willing to send any information i have to deal 
with
this issue..

Please provide: /var/log/kernel.log and /var/log/messages
It will be better if you can just compress and attach these logs instead of 
pasting
them here.

> and is it ok to use this on a dual-core system?
Yes it should be ok. Though, I have only ever tested on "virtual smp" system.

Original comment by nitingupta910@gmail.com on 5 Mar 2008 at 9:27

[GoogleCodeExporter]

excellent. I am running compcache-0.2 now on 3 systems. 

#1 is a AMD X2 Turion Socket S1 65nm SOI DDR2 2GB Laptop running Ubuntu 8.04 
Alpha-5
Kernel 2.6.24-11.29

#2 is an AMD64 Athlon Socket 939 90nm SOI DDR 2GB Desktop running Ubuntu 8.04 
Alpha-5
Kernel 2.6.24-11.29
#3 is an PXA270 130nm with 64MB's of SDRAM Zaurus SL-C1000 running Debian-Eabi 
Kernel
2.6.24-yonggun

I understand compcache is designed for embedded/swapless systems? is this 
correct? I
have plenty of memory on my full sized computers, I'd much rather have this 
running
on my zaurus than my laptop or desktop....

As soon as i crash i'll reboot and check out /var/log/kernel.log and
/var/log/messages , compress into .tar.bz2 format and attach. Is that the way 
to do it? 

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:37

[GoogleCodeExporter]

Compressed Caching for Linux
Description

Compressing swap pages and keeping them in RAM provides more memory space for
applications. This is especially useful for swapless embedded devices. Also, 
flash
storage typically used in embedded devices suffer from wear-leveling issues - 
so, its
very useful if we can avoid them using as swap device.

## This is especially useful for swapless embedded devices

so it should work on a PXA270 without any issues?

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:39

[GoogleCodeExporter]

i just wanted to mention again , it does take a long time to freeze. But then 
again
linux should NEVER freeze, so i know something is up.

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:40

[GoogleCodeExporter]

i noticed that often it freezes when installing packages through apt-get or 
just by
trying to make some sources.

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:41

[GoogleCodeExporter]

i dont think its system load that causes the freeze as much as something else in
apt-get specific packages like xxdiff or whatever. Its going to be a real eye 
opener
once i get to view these logs / messages..

Original comment by marblema...@gmail.com on 5 Mar 2008 at 9:44

[GoogleCodeExporter]

> I understand compcache is designed for embedded/swapless systems? is this 
correct?
Yes. And well...it should be useful for small mem desktops too - they usually 
have
decent processor.

> ## This is especially useful for swapless embedded devices
> so it should work on a PXA270 without any issues?

This is the intention :)

> As soon as i crash i'll reboot and check out /var/log/kernel.log and
>/var/log/messages , compress into .tar.bz2 format and attach. Is that the way 
to do
>it? 

Yes, this will be fine.

Original comment by nitingupta910@gmail.com on 5 Mar 2008 at 9:58

[GoogleCodeExporter]

> i noticed that often it freezes when installing packages through apt-get or 
just by
trying to make some sources.

Interesting. I think this is because apt-get or making sources is highly I/O 
bound
workload. When doing these things, you can try reducing compcache size to say 
10% of
RAM. For this, you'll have to reload compcache specifying the new size.

Original comment by nitingupta910@gmail.com on 5 Mar 2008 at 10:03

[GoogleCodeExporter]

i havent used more than 10% yet while testing, seems like its pretty stable as 
of
right now... but i shouldent speak too soon... Ill be posting soon enough with
results im sure, but i hope not. Maybe it works! lol

Original comment by marblema...@gmail.com on 5 Mar 2008 at 10:05

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Ive never seen any of these systems freeze before, unless they were running 
windows.
Linux is not supposed to freeze up like this. 6mb's of 64 should be OK?

Original comment by marblema...@gmail.com on 5 Mar 2008 at 10:09

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

does it make any difference that my ARM machine makes the compcache device at
/dev/compcache instead of /dev/compcache0?? would that passably freeze it?

Original comment by marblema...@gmail.com on 5 Mar 2008 at 11:16

[GoogleCodeExporter]

> does it make any difference that my ARM machine makes the compcache device at
> /dev/compcache instead of /dev/compcache0?? would that passably freeze it?

I don't understand how come its not called /dev/compcache0 on whatever machine. 
This
name (compcache0) is hardcoded in compcache-0.2 so the name must me same on 
whatever
machine you try. Are you using compcache-0.2?

Original comment by nitingupta910@gmail.com on 5 Mar 2008 at 11:22

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

not yet. I have to rebuild the kernel sources, then Ill use the latest svn and 
make
compcache-0.2. After that i hope everything goes well and the device name is
/dev/compcache0 instead of just /dev/compcache. I really think this project is 
just
what i need to take my machine to the next level in performance. I really have
enjoyed just the performance increase i get using compcache. I cant wait till i 
can
use it without crashing. Once i make the new modules i will hopefully not crash 
but
if i do Ill provide /var/log/kernel.log and /var/log/messages and hopefully we 
can
put this whole freezing thing behind us and enjoy fast embedded linux systems 
with
low amounts of ram. also using compressed ram is a hell of alot faster than 
using
swap partitions on a SDHC card, besides using SDHC cards as swap is a good way 
to
create bad blocks thus destroying the 16GB card.

Original comment by marblema...@gmail.com on 6 Mar 2008 at 12:14

[GoogleCodeExporter]

I am now using compcache-0.2 and its using /dev/compcache0... Looks pretty 
good, My
friend says it will freeze. I believe him, i just hope it doesnt. As soon as i 
freeze
i will post my logs...

Original comment by marblema...@gmail.com on 6 Mar 2008 at 1:50

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

already i can notice a really sweet speed up just with compiling the 2.6.24
sources... I love compcache!! embedded performance is important!!
Just when you think you need a ram upgrade and 64mb's of ram just aint enough 
for
your embedded linux box... compcache saves the day.. suddenly 64mb's of ram 
seems to
be enough to get all sorts of jobs done quickly! 

Original comment by marblema...@gmail.com on 6 Mar 2008 at 1:57

[GoogleCodeExporter]

*sigh* Frozen again.... Heres my kern.log and /var/log/messages.... I hope this 
fixes
the project... I want to have fun on my embedded device

Original comment by marblema...@gmail.com on 6 Mar 2008 at 2:10

Attachments:

• frozen-again-compcache-0.2-on-PXA270-2.6.24-yonggun.tar
• kernel.log
• messages

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

OK Im rebuilding my entire debian-eabi rootfs from scratch then rebuilding the 
kernel
sources 2.6.24 then ill make the latest SVN of compcache-0.2 and hope for the 
best..

if the same issue occurs ill post another set of log files just incase the 
output
isint the same. Thanks!

Original comment by marblema...@gmail.com on 6 Mar 2008 at 5:50

[GoogleCodeExporter]

Logs indicate that you are running out of memory and hence invoking oom-killer. 
This
almost surely looks like reason for system freeze. There is nothing in logs 
indicate
any bug in compcache-0.2 itself.

Also, you are using default compcache size with is 25% of RAM. So, it is 
setting ~15M 
 swap size on 64M system.

You can try setting compcache size to lower size, say 10%-15% of RAM. On your 
64M
system 10M compcache should work well.

do this to set compcache to 10M:
use_compcache.sh 10240

Original comment by nitingupta910@gmail.com on 6 Mar 2008 at 8:21

[GoogleCodeExporter]

alright ill try that exact setting when i load compache.ko using a brand new 
system
built from scratch with newly built sources. Hopefully everything will go well.

I have used 6mb's befor and also froze up using compcache-0.1... 

I pray this works. Ill try it tomorrow as soon as i wake up.. Ill leave the 
PXA270
overnight building the new sources then Ill make the latest modules and load 
them
using use_compcache.sh 10240

I hope it works

Original comment by marblema...@gmail.com on 6 Mar 2008 at 9:02

[GoogleCodeExporter]

> I have used 6mb's befor and also froze up using compcache-0.1... 

compcache-0.1 has a bug which is know to crash system. This bug was fixed in 
0.2.

Original comment by nitingupta910@gmail.com on 6 Mar 2008 at 9:07