Ray Serve lets users define an application builder function that constructs the Serve application. Users can then create a config file for the builder to deploy it to production.
The import path passed into serve build must be an Application object. If an application builder is passed in, serve build fails. serve build should be able to create config files for application builders.
Versions / Dependencies
Ray on the latest master.
Reproduction script
# File name: no_ops.py
import logging
from typing import Dict
from ray import serve
from ray.serve import Application
from ray.serve.handle import RayServeHandle
logger = logging.getLogger("ray.serve")
@serve.deployment
class Forward:
def __init__(self, handle: RayServeHandle):
self.handle = handle
async def __call__(self, *args, **kwargs):
return await (await self.handle.remote())
@serve.deployment
class NoOp:
def __call__(self):
return "No-op"
def app_builder(args: Dict[str, str]) -> Application:
valid_arg_keys = set(["num_forwards"])
if not valid_arg_keys.issuperset(args.keys()):
raise ValueError(
f"Got invalid args: {args.keys() - valid_arg_keys}. "
f"Valid args are {valid_arg_keys}."
)
num_forwards = int(args.get("num_forwards", 0))
if num_forwards == 0:
return NoOp.bind()
else:
app = Forward.bind(NoOp.bind())
for _ in range(num_forwards - 1):
app = Forward.bind(app)
return app
% serve build no_ops:app_builder
Traceback (most recent call last):
File "/Users/shrekris/miniforge3/envs/ae/bin/serve", line 33, in <module>
sys.exit(load_entry_point('ray', 'console_scripts', 'serve')())
File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/Users/shrekris/Desktop/ray/python/ray/serve/scripts.py", line 763, in build
build_app_config(import_path, False, f"app{app_index + 1}")
File "/Users/shrekris/Desktop/ray/python/ray/serve/scripts.py", line 709, in build_app_config
raise TypeError(
TypeError: Expected 'microbenchmarks.no_ops:app_builder' to be an Application but got <class 'function'>.
Issue Severity
Medium: It is a significant difficulty but I can work around it.
What happened + What you expected to happen
Ray Serve lets users define an application builder function that constructs the Serve application. Users can then create a config file for the builder to deploy it to production.
The import path passed into
serve build
must be anApplication
object. If an application builder is passed in,serve build
fails.serve build
should be able to create config files for application builders.Versions / Dependencies
Ray on the latest master.
Reproduction script
Issue Severity
Medium: It is a significant difficulty but I can work around it.