How to do server-sent events?

[Boscop]

How can I send server-sent events as a response from one of the worker routes? :)

(I was previously using nextjs with honojs in a cloudflare worker, and want to port my app to Rust/leptos.)

[DylanRJohnston]

Hey Boscop, unfortunately server sent events are not well suited to a serverless paradigm as you don't want the connection to keep your worker alive and costing you money. I think it's still very doable but would end up quite costly. I'd recommend making use of Durable Object's Hibernating Web Sockets feature instead https://developers.cloudflare.com/durable-objects/reference/websockets/#websocket-hibernation

[DylanRJohnston]

But if you wanted to go down this path you would just need to find some support for server sent events with axum and then it should all work out of the box. You can find an example here https://github.com/tokio-rs/axum/blob/main/examples/sse/src/main.rs

[DylanRJohnston]

For example I'm using hibernating web sockets with a game I'm working on to stream an event log to the game clients https://github.com/DylanRJohnston/deep-space-derby/blob/main/app/src/adapters/game_state/durable_object.rs

[DylanRJohnston]

The key parts for working with the websockets are https://github.com/DylanRJohnston/deep-space-derby/blob/84636992f000118548b6b028269f88b8baf03e18/app/src/adapters/game_state/durable_object.rs#L72-L90

[DylanRJohnston]

don't want the connection to keep your worker alive and costing you money

That being said, workers are billed on cpu time, not wall time. So perhaps it wouldn't end up costing so much money. You might run into problems with the maximum amount of time the worker is allowed to stay active though.

[Boscop]

@DylanRJohnston Thanks for the fast reply :) My use case is streaming LLM (json) responses incrementally (forwarding the streamed response), it's very important for letting the user already see the beginning of a response before it's fully generated. And yeah, workers are billed on CPU time, also the full response is completed after several seconds, so it won't be streaming forever..

So if I want to use SSE, do I have to use this? https://github.com/logankeenan/axum-cloudflare-adapter

[DylanRJohnston]

Not at all, Axum is fully supported by worker-rs that package predates Axum support. If you just use the Axum implementation of server side events it should work out of the box with this example repository.

[Boscop]

Ah thanks. And in which situation would axum-cloudflare-adapter be necessary?

[DylanRJohnston]

Never, it predates the support of Axum types directly in the worker-rs package. AFIAK that package should be archived

[DylanRJohnston]

You can check out this example for how to use server side event with axum https://github.com/tokio-rs/axum/blob/main/examples/sse/src/main.rs

async fn sse_handler(
    TypedHeader(user_agent): TypedHeader<headers::UserAgent>,
) -> Sse<impl Stream<Item = Result<Event, Infallible>>> {
    println!("`{}` connected", user_agent.as_str());

    // A `Stream` that repeats an event every second
    //
    // You can also create streams from tokio channels using the wrappers in
    // https://docs.rs/tokio-stream
    let stream = stream::repeat_with(|| Event::default().data("hi!"))
        .map(Ok)
        .throttle(Duration::from_secs(1));

    Sse::new(stream).keep_alive(
        axum::response::sse::KeepAlive::new()
            .interval(Duration::from_secs(1))
            .text("keep-alive-text"),
    )
}

[Boscop]

Thanks :)