Closed githuberj closed 1 month ago
Hello, see this related issue, but in general Ray serve is not expected to deliver high qps like normal FastAPI as it has a single point of ingress (proxy actor) that brings additional overhead, but I expect the overhead can be lowered in the future.
Thank you for the response. I will close the issue.
What happened + What you expected to happen
I have an issue or maybe just a wrong expectation while working with ray serve . I thought it might be able to run our whole fastapi application but it looks like it might be 10x slower that a normal fastapi application. Therefore I did this small benchmark. Maybe the benchmark is too simplistic. Could you provide a more comprehensive example then? I would just like to have our expectations set: Is it a bad idea to put general application logic into Ray?
Versions / Dependencies
ray[serve] == 2.32.0 fastapi == 0.111.1
Reproduction script
I used docker to test it and provided a zip file. benchmark.zip
Issue Severity
Low: It annoys or frustrates me.