search

heroku / buildpacks-nodejs

Heroku's Cloud Native Buildpacks for Node.js applications.
BSD 3-Clause "New" or "Revised" License
2 stars 1 forks source link

WEB_CONCURRENCY is miscalculated on ECS Fargate #220

Open ipmb opened 2 years ago

ipmb commented 2 years ago

Splitting this out from #210... When running on Fargate with 512MB of RAM, WEB_CONCURRENCY is set to 28. My expectation is that it would be set to 1 (or not set at all).

It looks like the calculations are taken from the following files. Their values on Fargate are:

$ cat /sys/fs/cgroup/memory/memory.limit_in_bytes
9223372036854771712
$ cat /sys/fs/cgroup/memory/memory.max
cat: /sys/fs/cgroup/memory/memory.max: No such file or directory

It's not yet clear to me the best way to determine how much RAM is available in a Fargate container, but I have an open issue with AWS support and will update when I hear back.

joshwlewis commented 2 years ago

$ cat /sys/fs/cgroup/memory/memory.limit_in_bytes 9223372036854771712

That seems like a bit more than 512MB! For what it's worth, I think dokku setups return a similarly large value. I would be interested to hear if there is a better way to arrive at this number on fargate.

ipmb commented 2 years ago

Fargate lets you set memory in two places, the "task" and the "container". If you set the container memory, this calculation works as expected. If you only set the task memory, there's no way (short of the API) to determine the value from inside the container.

edmorley commented 7 months ago

That seems like a bit more than 512MB! For what it's worth, I think dokku setups return a similarly large value. I would be interested to hear if there is a better way to arrive at this number on fargate.

Yeah I dug into this more for https://github.com/heroku/heroku-buildpack-python/pull/1547 - it turns out for cgroups v1 (older Docker and others) if there is no container memory limit is set, then instead of the /sys/fs/cgroup/memory/memory.limit_in_bytes file being missing, it's created with a nonsensically high value instead. In contrast for cgroups v2, if no limit is set, then the /sys/fs/cgroup/memory.max file instead contains the value "max".

Since this issue was opened, the implementation for heroku/nodejs-engine has been changed to now use a cap: https://github.com/heroku/buildpacks-nodejs/pull/764 (However, this still gives the wrong value for the Fargate use-case I believe?)