ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
33.99k stars 5.77k forks source link

[Serve] `serve build` fails with application builders #37819

Open shrekris-anyscale opened 1 year ago

shrekris-anyscale commented 1 year ago

What happened + What you expected to happen

Ray Serve lets users define an application builder function that constructs the Serve application. Users can then create a config file for the builder to deploy it to production.

The import path passed into serve build must be an Application object. If an application builder is passed in, serve build fails. serve build should be able to create config files for application builders.

Versions / Dependencies

Ray on the latest master.

Reproduction script

# File name: no_ops.py
import logging
from typing import Dict

from ray import serve
from ray.serve import Application
from ray.serve.handle import RayServeHandle

logger = logging.getLogger("ray.serve")

@serve.deployment
class Forward:
    def __init__(self, handle: RayServeHandle):
        self.handle = handle

    async def __call__(self, *args, **kwargs):
        return await (await self.handle.remote())

@serve.deployment
class NoOp:
    def __call__(self):
        return "No-op"

def app_builder(args: Dict[str, str]) -> Application:
    valid_arg_keys = set(["num_forwards"])
    if not valid_arg_keys.issuperset(args.keys()):
        raise ValueError(
            f"Got invalid args: {args.keys() - valid_arg_keys}. "
            f"Valid args are {valid_arg_keys}."
        )
    num_forwards = int(args.get("num_forwards", 0))
    if num_forwards == 0:
        return NoOp.bind()
    else:
        app = Forward.bind(NoOp.bind())
        for _ in range(num_forwards - 1):
            app = Forward.bind(app)
        return app
% serve build no_ops:app_builder            
Traceback (most recent call last):
  File "/Users/shrekris/miniforge3/envs/ae/bin/serve", line 33, in <module>
    sys.exit(load_entry_point('ray', 'console_scripts', 'serve')())
  File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/shrekris/miniforge3/envs/ae/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/Users/shrekris/Desktop/ray/python/ray/serve/scripts.py", line 763, in build
    build_app_config(import_path, False, f"app{app_index + 1}")
  File "/Users/shrekris/Desktop/ray/python/ray/serve/scripts.py", line 709, in build_app_config
    raise TypeError(
TypeError: Expected 'microbenchmarks.no_ops:app_builder' to be an Application but got <class 'function'>.

Issue Severity

Medium: It is a significant difficulty but I can work around it.

GeneDer commented 1 year ago

@zcin low priority, but when you have a sec, can you take a look? Thanks!