RC restarting for an unknown reason

[edzillion]


15:04:38
2018-09-27T15:04:38Z [INFO] DockerGoClient: process within container fbf6a340f597f76b7b8781638992119f793268749e3a300c9c0e63777fe0c094 (name: "ecs-rocketchat-td-199-rocketchat-beb8fdace3cee3e5bc01") died due to OOM

15:04:38
2018-09-27T15:04:38Z [WARN] Managed task [arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd]: 'docker stop' for container [rocketchat] returned OutOfMemoryError: Container killed due to memory usage

15:04:38
2018-09-27T15:04:38Z [INFO] Task [arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd]: recording execution stopped time. Essential container [rocketchat] stopped at: 2018-09-27 15:04:38.760051991 +0000 UTC m=+4158954.833013963

15:04:38
2018-09-27T15:04:38Z [INFO] Managed task [arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd]: sending container change event [rocketchat]: arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd rocketchat -> STOPPED, Exit 137, , Reason OutOfMemoryError: Container killed due to memory usage, Ports [{3000 80 0.0.0.0 0}], Known Sent: RUNNING

15:04:38
2018-09-27T15:04:38Z [INFO] Managed task [arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd]: sent container change event [rocketchat]: arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd rocketchat -> STOPPED, Exit 137, , Reason OutOfMemoryError: Container killed due to memory usage, Ports [{3000 80 0.0.0.0 0}], Known Sent: RUNNING

15:04:38
2018-09-27T15:04:38Z [INFO] Managed task [arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd]: sending task change event [arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd -> STOPPED, Known Sent: RUNNING, PullStartedAt: 2018-08-28 21:15:41.383301277 +0000 UTC m=+1589217.875300377, PullStoppedAt: 2018-08-28 21:15:42.70472337 +0000 UTC m=+15

15:04:38
2018-09-27T15:04:38Z [INFO] TaskHandler: batching container event: arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd rocketchat -> STOPPED, Exit 137, , Reason OutOfMemoryError: Container killed due to memory usage, Ports [{3000 80 0.0.0.0 0}], Known Sent: RUNNING

15:04:38
2018-09-27T15:04:38Z [INFO] TaskHandler: Adding event: TaskChange: [arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd -> STOPPED, Known Sent: RUNNING, PullStartedAt: 2018-08-28 21:15:41.383301277 +0000 UTC m=+1589217.875300377, PullStoppedAt: 2018-08-28 21:15:42.70472337 +0000 UTC m=+1589219.196722422, ExecutionStoppedAt: 2018-09-27 15:04:38.760051991 +0000 UTC m=+415

15:04:38
2018-09-27T15:04:38Z [INFO] TaskHandler: Sending task change: TaskChange: [arn:aws:ecs:eu-central-1:183869895864:task/d111ac45-72b2-4b3d-80e2-c8a9f7de76cd -> STOPPED, Known Sent: RUNNING, PullStartedAt: 2018-08-28 21:15:41.383301277 +0000 UTC m=+1589217.875300377, PullStoppedAt: 2018-08-28 21:15:42.70472337 +0000 UTC m=+1589219.196722422, ExecutionStoppedAt: 2018-09-27 15:04:38.760051991 +0000 UTC

15:04:38
2018-09-27T15:04:38Z [INFO] DockerGoClient: Unable to retrieve stats for container fbf6a340f597f76b7b8781638992119f793268749e3a300c9c0e63777fe0c094: context canceled

Seems like it is a mem issue. @xwvvvvwx what do you think, is this normal? The current settings are:

    "cpu": 256,
    "essential": true,
    "image": "rocketchat/rocket.chat:latest",
    "memory": 512,

I read somewhere on the RC docs that this should be sufficient, but tbh I don't really understand how it is allocated by AWS

[d-xo]

yeah looks like a memory issue to me as well. I would just bump the mem to 1 gig and see if it happens again 🤷‍♀️

[edzillion]

This was resolved after upping the memory on RC instances.