Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
With the migration of Ray to the new version 2.7.x, I am testing using FastAPI.
Before the problem: every argument that comes via POST request is now being converted into ray.serve.api.Ingress object at 0x. This prevents me from unpacking the JSON data in the first function. This object does not have methods like await or work with ray.get(myobject).
The simple way out would be to convert from async def to def.
I discovered that if I use a FastAPI Form for each function parameter, I can receive the data correctly.
When I add a type validation that has two options like str | list[str], the function returns only the 1st value in the list. To work the entire list I have to specify just list[str]. If I specify that the parameter can be str | None, and not passing this parameter I get error.
If I create a Pydantic Class for a function parameter I start to receive an error:
What happened + What you expected to happen
With the migration of Ray to the new version 2.7.x, I am testing using FastAPI.
Before the problem: every argument that comes via POST request is now being converted into
ray.serve.api.Ingress object at 0x
. This prevents me from unpacking the JSON data in the first function. This object does not have methods likeawait
or work with ray.get(myobject).The simple way out would be to convert from
async def
todef
.I discovered that if I use a FastAPI Form for each function parameter, I can receive the data correctly.
When I add a type validation that has two options like
str | list[str]
, the function returns only the 1st value in the list. To work the entire list I have to specify justlist[str]
. If I specify that the parameter can be str | None, and not passing this parameter I get error.If I create a Pydantic Class for a function parameter I start to receive an error:
{"detail":[{"loc":["body","prompt"],"msg":"field required","type":"value_error.missing"},{"loc":["body" ,"model"],"msg":"field required","type":"value_error.missing"}]}
Versions / Dependencies
Ray v2.7.1 Python 3.10 Ubuntu 22.04.2 LTS
fastapi v0.104.0 pydantic v1.10.13
Reproduction script
1. Multi string input. Error: value_error.missing
code
test
error
If I remove
self
from the function, the error remains2. Success with One parameter. Obtain object ref. But if add
self
returns same errorcode
test
shell
function with
self
error
3. Form use, it works
code
test
shell
4. Error: Union hint
code
test
error
using Union Typing operator does not change the error
5. If str or list str, get only last list value
code
test
shell
Issue Severity
High: It blocks me from completing my task.