Open aRyBernAlTEglOTRO opened 1 week ago
I think the issue is caused by the limitation of max_concurrency
in Actor, which default is 1000. A quick solution is to modify add the "max_concurrency" in allowed_ray_actor_options
in following and script:
and modify the RayActorOptionsSchema
in the following script to add the support for max_concurrency
.
But I think a better way is to align the max_ongoing_requests
in DeploymentConfig
and max_concurrency
in ray actor, because they seems like share the same intention, but it will need more code changes.
What happened + What you expected to happen
max_ongoing_requests
params in @serve.deployment isn't useful when it larger than 1000.max_ongoing_requests
is useful even it larger than 1000.Versions / Dependencies
Reproduction script
Reproducible Script:
Expect Output:
Actual Output:
Issue Severity
Low: It annoys or frustrates me.