search

ClosestStorm / rar2fs

Automatically exported from code.google.com/p/rar2fs
GNU General Public License v3.0
0 stars 0 forks source link

Out of memory #37

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Running a Windows 2008r2 server with a virtualbox guest (Ubuntu 14 server) 
running rar2fs.
Plex media server is running on the windows 2008r2 server and using the rar2fs 
server to unpack my movies and series. 

Having issue with rar2fs getting following message after X amount of data 
transfered: 

rar2fs login: Out of memory: Kill process 1046 (rar2fs) score 858 or sacrifice 
child 
Killed process 1046(rar2fs) total-vm:2409464kB  anon-rss:1760124kB, file-rss:0kB

When this happens i must run umount -l and remount it again.

I'm running 2 mounts from rar2fs, one with movies (with --seek-length=1) that 
has no issue so far. 
The other one(with the issue) does not have --seek-length=1 since it can't find 
all the rar files with it.

Tried increasing the memory from 512mb to 2gb, all it did was postpone the 
error bit longer - still happens thou.

Original issue reported on code.google.com by e...@vimmelman.se on 5 Sep 2014 at 9:04

GoogleCodeExporter commented 9 years ago 
Can you please tell what version of rar2fs you are using.

According to the error message rar2fs would have allocated ~2.5GiB of virtual 
memory. It obviously looks like a leak of some kind. I do not think it is 
directly related to running without seek-depth limit. How long does it take 
from mount to process death? Hours? Days?
One thing you could try is to only run the instance of rar2fs that is using no 
seek-depth limit. Does it still happen?
If you have top or htop, can you please let both instances run and 

"All modern Linux kernels have a built-in mechanism called “Out Of Memory 
killer” which can annihilate your processes under extremely low memory 
conditions. When such a condition is detected, the killer is activated and 
picks a process to kill. The target is picked using a set of heuristics scoring 
all processes and selecting the one with the worst score to kill."

That said, it might still not be rar2fs that is the root cause, even though it 
seems likely. But we need make sure before starting a ghost chase.

Original comment by hans.bec...@gmail.com on 6 Sep 2014 at 6:55

GoogleCodeExporter commented 9 years ago 
Something made the post to break up.

If you have top or htop, can you please let both instances run and make a 
couple of samples of VIRT, RES and SHR.

Original comment by hans.bec...@gmail.com on 6 Sep 2014 at 6:58

GoogleCodeExporter commented 9 years ago 
It happens periodically, but it feels like it happens a lot more when running 
more traffic through it.  For example when Plex server was running its initial  
sync/refresh it happened several times during a couple of hours. When the 
server wasn't in use for 2-3 days nothing happened.

Some more background info about how its setup:

Windows 2008r2 server running Virtualbox with:
Ubuntu 14.04 server (fully updated)  
1 CPU, 2Gb memory allocated.
rar2fs v1.20.0
unrarsrc 4.0.7
Windows folders shared to ubuntu machine through virtualbox shared folder
Unpacked folder shared back to Windows by samba

Running 2 folders with rar2fs. 
1 with Movies(with --seek-length=1) 
1 with Tv_series (with no seek-length)
Both folders are quite big (around 9TB for tv_series) but i thought 2Gb should 
suffice? 

When it crashes it only takes down tv_series, Movies never get affected. 

Tried today to run it with --seek-length=7 (no idea if this will help) 7 was 
the minimum to get most of the tv_series to unpack. 

Just ran a plex scan with htop running. Rar2fs (with TV_series) eats up more 
and more memory, when cancelling the scan - it never backs down.  Can send over 
some screen grabs by e-mail if interested

Original comment by e...@vimmelman.se on 6 Sep 2014 at 8:09

GoogleCodeExporter commented 9 years ago 
Since this is not going to be an easy task to trouble-shoot I will contact you 
by private e-mail. This will most likely require some iterations :(

Original comment by hans.bec...@gmail.com on 6 Sep 2014 at 8:23

GoogleCodeExporter commented 9 years ago 
Seems related to libunrar 4.0.7. Awaiting feedback from submitter how the 
system behaves after upgrade.

Original comment by hans.bec...@gmail.com on 8 Sep 2014 at 7:08

GoogleCodeExporter commented 9 years ago 
Can not reproduce the problem locally with the combination libunrar 4.0.7 and 
rar2fs 1.20.0. Upgrade to latest version of libunrar seems to resolve the 
reported issue. I will close this case until further information is provided or 
I receive any additional reports about libunrar 4.0.7 is causing problems

Original comment by hans.bec...@gmail.com on 15 Sep 2014 at 6:14

GoogleCodeExporter commented 9 years ago 
I am opening this issue again. A weakness in libunrar 4.0.7 has been 
identified. Since some of the library extension are based on the same code also 
rar2fs may suffer from the same potential memory leaks. This should not be a 
serious issue since it only affects some extreme error cases, but severe enough 
to investigate it a bit further.

Original comment by hans.bec...@gmail.com on 18 Sep 2014 at 9:41

GoogleCodeExporter commented 9 years ago 
Just for the record, the potential memory leak was fixed in libunrar 4.2.0 and 
later.

Original comment by hans.bec...@gmail.com on 18 Sep 2014 at 9:46

GoogleCodeExporter commented 9 years ago 
A potential fix for this issue has been committed to trunk.

Original comment by hans.bec...@gmail.com on 24 Oct 2014 at 8:03

GoogleCodeExporter commented 9 years ago 
Missing feedback from submitter about the latest patch.
Will close this issue since a weakness in libunrar 4.0.7 was confirmed and a 
workaround in rar2fs is committed to trunk.

Original comment by hans.bec...@gmail.com on 11 Nov 2014 at 8:56