search

project-everest / everest-ci

CI scripts for project everest
3 stars 8 forks source link

figure out if there's a way for Docker not to hog the RAM all the time #31

Closed msprotz closed 7 years ago

msprotz commented 7 years ago

Right now any RAM devoted to Docker cannot be reclaimed by other processes, as far as I understand

darrenge commented 7 years ago

I researched this a bit and it doesn't look like it is possible. I have some emails out to docker people and on the docker user forum. My guess is "no" since the settings page where you set the memory usage says "Adjust the computing resources dedicated to Docker.". That sounds like it is pretty dedicated. Will see if I can get official word on it though before closing this issue.

darrenge commented 7 years ago

Looking here: https://docs.docker.com/engine/reference/run/#/user-memory-constraints User Memory Constraints: memory=inf, memory-swap=inf (default) - There is no memory limit for the container. The container can use as much memory as needed. memory=L<inf, memory-swap=inf -- (specify memory and set memory-swap as -1) The container is not allowed to use more than L bytes of memory, but can use as much swap as is needed (if the host supports swap memory). memory=L<inf, memory-swap=2*L -- (specify memory without memory-swap) The container is not allowed to use more than L bytes of memory, swap plus memory usage is double of that. memory=L<inf, memory-swap=S<inf, L<=S -- (specify both memory and memory-swap) The container is not allowed to use more than L bytes of memory, swap plus memory usage is limited by S.

darrenge commented 7 years ago

My take on this is that we can limit the memory to a specific amount per container or on the docker agent for total memory usage. There isn't anything about having shared memory resources where Docker's allocated memory is available to other processes.

darrenge commented 7 years ago

Asked some docker experts and general consensus is "RAM devoted to Docker cannot be reclaimed by other processes"

msprotz commented 7 years ago

Thanks for looking into it. Moving forward, the tradeoff will be that we allocate a little bit of RAM for the Docker process (e.g. 16GB) but don't parallelize the build too much since the RAM can't scale up.