rancher / os

Tiny Linux distro that runs the entire OS as Docker containers
https://rancher.com/docs/os/v1.x/en/
Apache License 2.0
6.44k stars 655 forks source link

DO High CPU consumption - unable to login to RancherOS UI #3020

Open connde opened 4 years ago

connde commented 4 years ago

RancherOS Version: 1.5.6

Where are you running RancherOS? DigitalOcean

I'm trying to use RancherOS in DigitalOcean, which is pretty easy, but I noticed that after I installed longhorn my CPU is going to 100% and I cannot login anymore, I had to delete all resources.

Not sure if it is Longhorn, could someone help me finding out what is wrong?

Steps are basically creating a new droplets with RancherOS (1vcpu - 2gb ram), creating 3 nodes (2vpcu - 4gb ram - 60gb hdd) and installing Longhorn, without changing the basic settings. After some minutes it start consuming 100% and UI is not accessible anymore.

Restoring a snapshot does not works.

image

connde commented 4 years ago

Hi, this is actually not related to Longhorn, I just created new RancherOS droplet and new cluster with 3 nodes and I'm getting this, I thought 2gb and 1vcpu was enough but I'm trying to resize the VM to see if problem is solved.

` [rancher@rancheros-s-1vcpu-2gb-sfo2-01 ~]$ docker ps fatal error: runtime: out of memory

runtime stack: runtime.throw(0x1def39d, 0x16) /usr/local/go/src/runtime/panic.go:774 +0x72 fp=0x7ffe7863f448 sp=0x7ffe7863f418 pc=0x42e5a2 runtime.sysMap(0xc000000000, 0x4000000, 0x304af38) /usr/local/go/src/runtime/mem_linux.go:169 +0xc5 fp=0x7ffe7863f488 sp=0x7ffe7863f448 pc=0x418245 runtime.(mheap).sysAlloc(0x3031c40, 0x2000, 0x0, 0x0) /usr/local/go/src/runtime/malloc.go:701 +0x1cd fp=0x7ffe7863f530 sp=0x7ffe7863f488 pc=0x40b2ad runtime.(mheap).grow(0x3031c40, 0x1, 0xffffffff) /usr/local/go/src/runtime/mheap.go:1255 +0xa3 fp=0x7ffe7863f580 sp=0x7ffe7863f530 pc=0x4262f3 runtime.(mheap).allocSpanLocked(0x3031c40, 0x1, 0x304af48, 0x0) /usr/local/go/src/runtime/mheap.go:1170 +0x266 fp=0x7ffe7863f600 sp=0x7ffe7863f580 pc=0x426186 runtime.(mheap).alloc_m(0x3031c40, 0x1, 0x2a, 0x0) /usr/local/go/src/runtime/mheap.go:1022 +0xc2 fp=0x7ffe7863f650 sp=0x7ffe7863f600 pc=0x425a22 runtime.(mheap).alloc.func1() /usr/local/go/src/runtime/mheap.go:1093 +0x4c fp=0x7ffe7863f688 sp=0x7ffe7863f650 pc=0x4574ac runtime.(mheap).alloc(0x3031c40, 0x1, 0x1002a, 0x0) /usr/local/go/src/runtime/mheap.go:1092 +0x8a fp=0x7ffe7863f6d8 sp=0x7ffe7863f688 pc=0x425d0a runtime.(mcentral).grow(0x3032a78, 0x0) /usr/local/go/src/runtime/mcentral.go:255 +0x7b fp=0x7ffe7863f718 sp=0x7ffe7863f6d8 pc=0x417c6b runtime.(mcentral).cacheSpan(0x3032a78, 0x7fc5c5aa0000) /usr/local/go/src/runtime/mcentral.go:106 +0x2fe fp=0x7ffe7863f778 sp=0x7ffe7863f718 pc=0x41778e runtime.(mcache).refill(0x7fc5c5aa0008, 0x2a) /usr/local/go/src/runtime/mcache.go:138 +0x85 fp=0x7ffe7863f798 sp=0x7ffe7863f778 pc=0x417235 runtime.(mcache).nextFree(0x7fc5c5aa0008, 0x304af2a, 0x7fc5c5aa0008, 0x8, 0xfffffffffffffff8) /usr/local/go/src/runtime/malloc.go:854 +0x87 fp=0x7ffe7863f7d0 sp=0x7ffe7863f798 pc=0x40bad7 runtime.mallocgc(0x180, 0x1dc7240, 0x1, 0x304afa0) /usr/local/go/src/runtime/malloc.go:1022 +0x793 fp=0x7ffe7863f870 sp=0x7ffe7863f7d0 pc=0x40c413 runtime.newobject(0x1dc7240, 0x4000) /usr/local/go/src/runtime/malloc.go:1151 +0x38 fp=0x7ffe7863f8a0 sp=0x7ffe7863f870 pc=0x40c808 runtime.malg(0x4d0700008000, 0x30342e8) /usr/local/go/src/runtime/proc.go:3232 +0x31 fp=0x7ffe7863f8e0 sp=0x7ffe7863f8a0 pc=0x437941 runtime.mpreinit(...) /usr/local/go/src/runtime/os_linux.go:324 runtime.mcommoninit(0x302bf00) /usr/local/go/src/runtime/proc.go:626 +0xc2 fp=0x7ffe7863f918 sp=0x7ffe7863f8e0 pc=0x431362 runtime.schedinit() /usr/local/go/src/runtime/proc.go:543 +0x72 fp=0x7ffe7863f970 sp=0x7ffe7863f918 pc=0x430fc2 runtime.rt0_go(0x7ffe7863f9a8, 0x2, 0x7ffe7863f9a8, 0x0, 0x0, 0x2, 0x7ffe78640e94, 0x7ffe78640e9b, 0x0, 0x7ffe78640e9e, ...) /usr/local/go/src/runtime/asm_amd64.s:214 +0x125 fp=0x7ffe7863f978 sp=0x7ffe7863f970 pc=0x4598f5 `

connde commented 4 years ago

Looks like 3g is just enough but 4gb would be recommended to run RancherOS. I misunderstood the documentation that states 1gb? Resized to 1vpc-3gb image

Resized to 2vpc-4gb image