[Installation]: Docker runs out of CPU swap size on 8 GPUs. How to lower swap_space to be less than 4GB per GPU?

[elabz]

Your current environment

The output of `python env.py`

My apologies, everyone, not sure how to provide environment data from a docker container that cannot start. But the hw setup is:

32GB RAM 8 x 6GB VRAM (P106-100 GPUs)

Getting this error on trying to start docker container:

aphrodite-engine  | ValueError: Too large swap space. 32.00 GiB out of the 31.27 GiB total CPU memory is allocated for the swap space.

Please note that the docker run command in Readme did not work since it would not pass HF_TOKEN, and the model cannot be downloaded. So, I had to set values in .env and then do docker compose build and docker compose up in order for it to get as far as try to start.

I can see a reference to swap_space in aphrodite-engine/aphrodite/common/config.py but the value seems to be hardcoded at 4GB elsewhere. Is there a way to lower CPU RAM Per GPU requirement by passing a config value to the docker container? If so, how? Would much appreciate help in getting past this. Thanks!

How did you install Aphrodite?

Docker install from aphrodite-engine/docker

[elabz]

Also, starting it on 4 GPUs I get this warning, which makes sense, but how do I configure the size?

WARNING:  Possibly too large swap space. 16.00 GiB out of the 31.27 GiB total

[AlpinDale]

--swap-space should handle this