memory leak?

[GoogleCodeExporter]

On my s3backer testsystem that is backing up several gigabytes of photos to
my freshly created s3backer partition, I see a steady increase of memory
usage by s3backer.
I started monitoring this because the backup process kept crashing due to
the unavailability of the loopack mount, which was due to s3backer having
stopped see dmesg output (I'm not sure what the start of the error is)

...
nmbd invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
Pid: 3050, comm: nmbd Tainted: G        W  2.6.28.4 #11
Call Trace:
 [<c01345f6>] oom_kill_process+0x4d/0x17c
 [<c01349aa>] out_of_memory+0x133/0x15d
 [<c01364a4>] __alloc_pages_internal+0x2ce/0x373
 [<c0137af8>] __do_page_cache_readahead+0x74/0x152
 [<c0137ea6>] do_page_cache_readahead+0x3d/0x47
 [<c0133f6a>] filemap_fault+0x133/0x2e1
 [<c0123596>] __wake_up_bit+0x25/0x2a
 [<c013c44d>] __do_fault+0x3f/0x2da
 [<c013d617>] handle_mm_fault+0x205/0x423
 [<c01100ba>] do_page_fault+0x238/0x556
 [<c010fe82>] do_page_fault+0x0/0x556
 [<c03c31e2>] error_code+0x6a/0x70
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd: 130
Active_anon:42305 active_file:38 inactive_anon:42423
 inactive_file:767 unevictable:0 dirty:1 writeback:6 unstable:0
 free:1250 slab:2226 mapped:19 pagetables:617 bounce:0
DMA free:1828kB min:92kB low:112kB high:136kB active_anon:4372kB
inactive_anon:4604kB active_file:12kB inactive_file:216kB unevictable:0kB
present:15868kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 427 427
Normal free:3172kB min:2596kB low:3244kB high:3892kB active_anon:164848kB
inactive_anon:165088kB active_file:140kB inactive_file:2852kB
unevictable:0kB present:437768kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
DMA: 1*4kB 4*8kB 0*16kB 0*32kB 2*64kB 1*128kB 0*256kB 1*512kB 1*1024kB
0*2048kB 0*4096kB = 1828kB
Normal: 151*4kB 7*8kB 3*16kB 5*32kB 8*64kB 4*128kB 1*256kB 0*512kB 1*1024kB
0*2048kB 0*4096kB = 3172kB
826 total pagecache pages
0 pages in swap cache
Swap cache stats: add 890828, delete 890828, find 45044/67748
Free swap  = 0kB
Total swap = 610252kB
114400 pages RAM
2165 pages reserved
8404 pages shared
102350 pages non-shared
Out of memory: kill process 9655 (s3backer) score 15018 or a child
Killed process 9655 (s3backer)
Buffer I/O error on device loop0, logical block 261160960
...

After that I've watched cat /proc/`pidof s3backer`/status | grep Vm for
some time, and it shows the numbers steadily increasing over time.

I'm not a C debugger, so I may be completely wrong about a suspected leak,
but the fact remains that s3backer keeps crashing on me on this system.

This is the commandline I use for the s3backer mount:
s3backer --vhost --blockCacheFile=/var/cache/s3backer/s3b-cache
--blockCacheSize=256 ******* /mnt/s3backer

Original issue reported on code.google.com by mrva...@gmail.com on 15 Oct 2009 at 9:30

[GoogleCodeExporter]

Thanks for the report. I assume you are not doing anything special other that 
lots of
writes to the s3backer filesystem.

I will attempt to reproduce this on my system.

Original comment by archie.c...@gmail.com on 16 Oct 2009 at 1:45

• Changed state: Accepted
• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Well.. it's my media player, so it's doing a lot of mythtv as well, but 
concerning
s3backer it's only writing (rsync) to the s3backer mount (a lot), yes.

Original comment by mrva...@gmail.com on 16 Oct 2009 at 1:51

[GoogleCodeExporter]

This is the output of cat /proc/`pidof s3backer`/status | grep Vm after one day 
of
uploading (still going strong).

VmPeak:   646340 kB
VmSize:   645644 kB
VmLck:         0 kB
VmHWM:    264720 kB
VmRSS:    255168 kB
VmData:   639216 kB
VmStk:        84 kB
VmExe:        88 kB
VmLib:      5988 kB
VmPTE:       552 kB

VmHWM and VmRSS seem stable, but VmPeak, VmSize and VmData keep growing 
steadily a
couple of kb's every time I run the command.

Original comment by mrva...@gmail.com on 16 Oct 2009 at 2:46

[GoogleCodeExporter]

I don't have a definitive answer yet but do have one possible theory.

Please apply the attached patch ("nocancel.patch.txt") and let me know if this 
fixes
the problem for you.

Original comment by archie.c...@gmail.com on 16 Oct 2009 at 5:57

Attachments:

• nocancel.patch.txt

[GoogleCodeExporter]

Found the problem. Fixed in r400.

Original comment by archie.c...@gmail.com on 16 Oct 2009 at 7:04

• Changed state: Fixed
• Added labels: AffectsVersion-1.3.0, FixVersion-1.3.1

[GoogleCodeExporter]

As far as I can tell from the little testing I've done, I think it's fixed 
indeed. Thx!

Original comment by mrva...@gmail.com on 17 Oct 2009 at 10:36