My docker-compose.yml defines the shm_size: '32gb' but when I run ray.init(num_cpus=4, ignore_reinit_error=True, include_dashboard=False) I get the log line:
2020-08-26 20:31:09,651 INFO resource_spec.py:231 -- Starting Ray with 37.5 GiB memory available for workers and up to 18.76 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).
So I am naturally curious:
How is ray getting more memory than is allotted to the container in which it's running?
My
docker-compose.ymldefines theshm_size: '32gb'but when I runray.init(num_cpus=4, ignore_reinit_error=True, include_dashboard=False)I get the log line:2020-08-26 20:31:09,651 INFO resource_spec.py:231 -- Starting Ray with 37.5 GiB memory available for workers and up to 18.76 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).So I am naturally curious:
How is
raygetting more memory than is allotted to the container in which it's running?Thanks in advance