ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
33k stars 5.58k forks source link

[Serve] Rebase Serve to become transparent HTTP/2 proxy to avoid parsing gRPC payloads #46174

Open alexeykudinkin opened 2 months ago

alexeykudinkin commented 2 months ago

What happened + What you expected to happen

Currently using gRPC with Serve entails

Instead, we'd just rely on server (uvicorn) to transparently proxy on the application transport layer (HTTP/2) w/o proxy meddling as man-in-the-middle.

Versions / Dependencies

2.22

Reproduction script

N/A

Issue Severity

Medium: It is a significant difficulty but I can work around it.

alexeykudinkin commented 2 months ago

Seems like uvicorn is actually having no plans to support H2 and we might need to move to hypercorn: https://github.com/encode/uvicorn/issues/47#issuecomment-1029020324

zcin commented 2 months ago

@GeneDer could you help triage this

GeneDer commented 2 months ago

yea, sorry didn't know this had the triage tag