[Serve] does not work with tracing

[judahrand]

What happened + What you expected to happen

When enabling tracing on a Ray Serve deployment I get an error:

(ProxyActor pid=71635) WARNING 2024-06-25 17:52:09,114 proxy 127.0.0.1 pow_2_scheduler.py:536 - Failed to fetch queue length for Replica(id='ukdreh12', deployment='Deployment', app='default'): 'got an unexpected keyword argument '_ray_trace_ctx''

I'd hoped that tracing would just work with Ray Serve so that I could set up OTEL traces.

Versions / Dependencies

ray==2.30.0

Reproduction script

from typing import Any

import ray
import ray.serve
import requests

ray.init(
    _tracing_startup_hook='ray.util.tracing.setup_local_tmp_tracing:setup_tracing',
)

@ray.serve.deployment
class Deployment:
    def __call__(self, *args: Any, **kwargs: Any) -> str:
        return 'Hello, world!'

app = Deployment.bind()
ray.serve.run(app)

requests.post('http://localhost:8000', json={})

Issue Severity

High: It blocks me from completing my task.

[anyscalesam]

@judahrand - yeah we're aware - working on restoring Open Telemetry working again. Would you be interested in contributing; we're working on an RFC that we'll publish but need help in implementing.