Panic on running out of memory on ASUS RT-AC68R router

[tydynrain]

I've installed Syncthing on my ASUS AC-68R router running ASUSWRT-Merlin firmware under an entware environment. I've gotten it mostly working, howerver, given that this router only has 250MB of RAM, no matter what I do, Syncthing runs out of memory every single time and dies, usually in about 5-10 minutes. I've attempted to limit its resource usage using GOGC, without success.

Here is one of the numerous terminal outputs as it begins to panic and die:

fatal error: runtime: out of memory [monitor] 05:24:47 WARNING: Panic detected, writing to "/root/.config/syncthing/panic-20160428-052447.log" [monitor] 05:24:47 WARNING: Please check for existing issues with similar panic message at https://github.com/syncthing/syncthing/issues/ [monitor] 05:24:47 WARNING: If no issue with similar panic message exists, please create a new issue with the panic log attached fatal error: runtime: out of memory

runtime stack: runtime.throw(0x6c2af0, 0x16) /usr/local/go/src/runtime/panic.go:530 +0x78 runtime.sysMap(0x10f00000, 0x100000, 0x1, 0xacc9b8) /usr/local/go/src/runtime/mem_linux.go:217 +0x15c runtime.(_mheap).sysAlloc(0xabe228, 0x100000, 0xf) /usr/local/go/src/runtime/malloc.go:429 +0x13c runtime.(_mheap).grow(0xabe228, 0x8, 0x0) /usr/local/go/src/runtime/mheap.go:651 +0x68 runtime.(_mheap).allocSpanLocked(0xabe228, 0x1, 0x0) /usr/local/go/src/runtime/mheap.go:553 +0x5b4 runtime.(_mheap).alloc_m(0xabe228, 0x1, 0xe, 0x10d31e00, 0x6b870) /usr/local/go/src/runtime/mheap.go:437 +0x138 runtime.(_mheap).alloc.func1() /usr/local/go/src/runtime/mheap.go:502 +0x40 runtime.systemstack(0x10d31f08) /usr/local/go/src/runtime/asm_arm.s:261 +0xa8 runtime.(_mheap).alloc(0xabe228, 0x1, 0xe, 0x100, 0x1) /usr/local/go/src/runtime/mheap.go:503 +0x58 runtime.(_mcentral).grow(0xabefc8, 0x0) /usr/local/go/src/runtime/mcentral.go:209 +0xb0 runtime.(_mcentral).cacheSpan(0xabefc8, 0x10d55b9c) /usr/local/go/src/runtime/mcentral.go:89 +0x5a8 runtime.(*mcache).refill(0x40079258, 0xe, 0x10d55b40) /usr/local/go/src/runtime/mcache.go:119 +0xd4 runtime.mallocgc.func2() /usr/local/go/src/runtime/malloc.go:642 +0x28 runtime.systemstack(0x10d16000) /usr/local/go/src/runtime/asm_arm.s:247 +0x80 runtime.mstart() /usr/local/go/src/runtime/proc.go:1048

goroutine 32 [running]: runtime.systemstack_switch() /usr/local/go/src/runtime/asm_arm.s:192 +0x4 fp=0x10d3acd4 sp=0x10d3acd0 runtime.mallocgc(0xd0, 0x0, 0x3, 0x0) /usr/local/go/src/runtime/malloc.go:643 +0x930 fp=0x10d3ad44 sp=0x10d3acd4 runtime.rawstring(0xc2, 0x0, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/runtime/string.go:284 +0x64 fp=0x10d3ad68 sp=0x10d3ad44 runtime.rawstringtmp(0x0, 0xc2, 0x0, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/runtime/string.go:111 +0xb4 fp=0x10d3ad84 sp=0x10d3ad68 runtime.slicebytetostring(0x0, 0x10d5fee0, 0xc2, 0xc2, 0x0, 0x0) /usr/local/go/src/runtime/string.go:93 +0x48 fp=0x10d3add0 sp=0x10d3ad84 bufio.(*Reader).ReadString(0x10d3af4c, 0x10ee460a, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/bufio/bufio.go:447 +0x84 fp=0x10d3adfc sp=0x10d3add0 main.copyStderr(0x4023e120, 0x10d4eba0, 0x4023e1c0, 0x10d4e0a0) /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:173 +0xb4 fp=0x10d3afa4 sp=0x10d3adfc main.monitorMain.func1(0x401c9030, 0x10d4eba0, 0x10d4e990, 0x401c9050, 0x10e58b40) /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:113 +0x60 fp=0x10d3afbc sp=0x10d3afa4 runtime.goexit() /usr/local/go/src/runtime/asm_arm.s:990 +0x4 fp=0x10d3afbc sp=0x10d3afbc created by main.monitorMain /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:115 +0xc08

goroutine 1 [select]: main.monitorMain(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x65fc7c, 0x1, 0x0, ...) /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:130 +0x1658 main.main() /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/main.go:366 +0x75c

goroutine 20 [syscall]: os/signal.signal_recv(0x0) /usr/local/go/src/runtime/sigqueue.go:116 +0x190 os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:22 +0x14 created by os/signal.init.1 /usr/local/go/src/os/signal/signal_unix.go:28 +0x30

goroutine 21 [chan receive]: main.trackCPUUsage() /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/gui_unix.go:24 +0x108 created by main.init.3 /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/gui_unix.go:17 +0x24

goroutine 22 [select, locked to thread]: runtime.gopark(0x780648, 0x10d1af8c, 0x6681a8, 0x6, 0xabb318, 0x2) /usr/local/go/src/runtime/proc.go:262 +0x148 runtime.selectgoImpl(0x10d1af8c, 0x0, 0xc) /usr/local/go/src/runtime/select.go:392 +0x1204 runtime.selectgo(0x10d1af8c) /usr/local/go/src/runtime/select.go:215 +0x10 runtime.ensureSigM.func1() /usr/local/go/src/runtime/signal1_unix.go:279 +0x428 runtime.goexit() /usr/local/go/src/runtime/asm_arm.s:990 +0x4

goroutine 33 [syscall]: syscall.Syscall(0x3, 0xa, 0x10efe000, 0x1000, 0x43, 0x0, 0x0) /usr/local/go/src/syscall/asm_linux_arm.s:17 +0x8 syscall.read(0xa, 0x10efe000, 0x1000, 0x1000, 0x43, 0x0, 0x0) /usr/local/go/src/syscall/zsyscall_linux_arm.go:783 +0x78 syscall.Read(0xa, 0x10efe000, 0x1000, 0x1000, 0x43, 0x0, 0x0) /usr/local/go/src/syscall/syscall_unix.go:161 +0x4c os.(_File).read(0x10d4ebc0, 0x10efe000, 0x1000, 0x1000, 0x43, 0x0, 0x0) /usr/local/go/src/os/file_unix.go:228 +0x54 os.(_File).Read(0x10d4ebc0, 0x10efe000, 0x1000, 0x1000, 0x43, 0x0, 0x0) /usr/local/go/src/os/file.go:95 +0x7c bufio.(_Reader).fill(0x10d1bf54) /usr/local/go/src/bufio/bufio.go:97 +0x1c4 bufio.(_Reader).ReadSlice(0x10d1bf54, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/bufio/bufio.go:328 +0x264 bufio.(_Reader).ReadBytes(0x10d1bf54, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/bufio/bufio.go:406 +0x7c bufio.(_Reader).ReadString(0x10d1bf54, 0x10ef270a, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/bufio/bufio.go:446 +0x44 main.copyStdout(0x4023e120, 0x10d4ebc0, 0x4023e1c0, 0x10d4e0a0) /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:215 +0xac main.monitorMain.func2(0x401c9030, 0x10d4ebc0, 0x10d4e990, 0x401c9050, 0x10e58b40) /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:119 +0x60 created by main.monitorMain /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:121 +0xc60

goroutine 34 [semacquire]: sync.runtime_Semacquire(0x10e58b4c) /usr/local/go/src/runtime/sema.go:47 +0x24 sync.(*WaitGroup).Wait(0x10e58b40) /usr/local/go/src/sync/waitgroup.go:127 +0xc0 main.monitorMain.func3(0x401c9050, 0x10e58b40, 0x10ef53c0, 0x10ee43c0) /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:126 +0x24 created by main.monitorMain /home/buildslave/workspace/syncthing-release/src/github.com/syncthing/syncthing/cmd/syncthing/monitor.go:128 +0xcb4

This is my first time submitting, so please forgive me if I did so incorrectly. It's late, and I'm very tired of fighting with Syncthing and my router to get things working. Unless the memory issue is rectified, I don't see how I will be able to make use of Syncthing for in this context.

[calmh]

Your box has too little memory to run syncthing with the configuration you have for it. I don't see a way around that. Sorry.

[danielbodart]

I had exactly the same problem as @tydynrain on my ASUS DSL-AC68U with 256MB of ram. I noticed it wasn't actually running out of memory (21MB free) when it crashed which made me suspicious and I fixed the problem by just creating a swap partition on my usb connected hard drive and then turning swapon /dev/sda1 (replace with your correct partition etc), I stuffed this into the usb mount script in the nvram.

Doesn't even need to be very big as it only used an extra 10MB of swap while indexing.

I think this is a GO bug by the way as I had exactly the same problem with btsync