[docs][core] Missing guidance on physical memory model in Ray - Githubissues

ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

https://ray.io

Apache License 2.0

33.92k stars 5.77k forks source link

[docs][core] Missing guidance on physical memory model in Ray #25890

Open stephanie-wang opened 2 years ago

stephanie-wang commented 2 years ago

Description

We have a lot of different types of memory in Ray:

Plasma shared memory
Local disk / external storage for spilling
Fallback allocation to /tmp
Heap memory (which varies depending on whether shared memory objects support zero-copy reads or not)

I looked in the docs but I don't see a definitive guide on how these relate to each other, and common performance gotchas like:

Calling ray.get on too many objects at a time
What to do if there is unexpected spilling
How to diagnose and prevent worker heap memory OOMs

cc @ericl @scv119

Link

No response

rkooo567 commented 2 years ago

cc @jjyao