-
### 🚀 The feature, motivation and pitch
Currently, distributed inference (TP) in vLLM relies on ray to orchestrate the gpu workers. I briefly check the code and seems the core distributed communica…
-
Forgive my ignorance.
I didn't quite understand the configuration of the program
1. Start two dockers in learnring and action respectively? and then populate action.py with the docker IP address for…
-
This has an email thread where we dive into details. Awaiting a conclusion there.
-
The readme talks about remote actors, which looks super interesting! Are there more details on how that would work?
One usage example would be to build distributed apps with one side being a server…
-
### Rationale:
I'd generally like to include docs validation in CI, and require "no warnings".
### Problem:
Can't do that easily since my dependencies sometimes have warnings themselfes. E.g…
ktoso updated
2 years ago
-
Correct me if I am wrong, but I believe right now, the meta node tries to schedule the actors/parallel units(not sure which is the correct terminology) of a streaming job to all the CNs as evenly as p…
lmatz updated
1 month ago
-
Building blocks:
- 🟢 Service invocation
- [x] Implementation
- [x] Docs #126
- [x] Validation/Example(s)
- 🟠 State management
- [x] Implementation [TBA](#)
- [x] Docs #126
- [ ] …
-
First off, thank you all for this great framework!
I ran into the following issue:
I want to distribute actors which consume a significant amount of RAM. For instance, each worker holds a large Nu…
-
## In what area(s)?
/area runtime
## Describe the feature
Today, actor reminders are scheduled using a timespan; that is, how long from now should the reminder be triggered. This is generally fin…
-
At https://www.interance.io/learning, choose to view article "Distributed Actors", but go to article "Monitoring and Linking".