Add debouncing at the right place

[seefeldb]

maybe generateData?

key is that it is per node. at first maybe just allow only one request to be in-flight.

alternative: a debounce node that copies a value with some delay (drawback: it doesn't know about the request downstream). maybe both?

[gordonbrander]

In my experience sinks are usually right place to apply debouncing for reactive cells.

• You usually want granular updates for data updates. Batching data updates can cause unexpected behavior unless the batching semantics are well-defined and universal.
• The perf bottleneck is typically DOM writes.
• Batching at sinks lets you choose your update sematics (e.g. RAF for DOM).

[seefeldb]

good framing. the context here was expensive LLM calls. what's the equivalent of the sink here? the point where the network call is made?

[gordonbrander]

Yeah probably at the interface where the cell meets the service.

It could also be fine to have something like a "debounce propagator" that gives you a debounced derivative cell, but it's probably something you want to opt into as a transformation, rather than being part of the low-level cell semantics.

[anotherjesse]

https://github.com/anthropics/anthropic-sdk-typescript?tab=readme-ov-file#streaming-responses

to add more complexity - we do have some ability to cancel in-progress streams (and only be billed for the processed/generated tokens)

[gordonbrander]

I think I might try to capture this cancellation semantics in the cell value. E.g.

• include a request ID in the cell data.
• track streams by request ID
• Cancel existing streams when request ID changes

[anotherjesse]

can we trust tcp / canceling a fetch to be a way to communicate with "planning server" that it should cancel the stream?

or should we move to websockets to be able to say "belay that order!"

[seefeldb]

I think I might try to capture this cancellation semantics in the cell value. E.g.

• include a request ID in the cell data.
• track streams by request ID
• Cancel existing streams when request ID changes

This is almost what the current semantics are, and they are compatible with what @Gozala suggests for effects like this:

We currently write a requesr into a cell and the fetch node reacts to those (in @Gozala 's version that node just returns an entity that represents the http request, which then another service queries for). It should stop the current request when this changes and then maybe further denounce subsequent ones. It writes pending, result (undefined until the request is done), partial (the stream so far) and error. And it should also write the hash of the request into the output once done. The latter is mostly for itself, and it'll read it back in, so that on rehydration it'll know to not send the request again, since the result already corresponds to the request.

We can add additional denouncing before that to delay the first request. But if so we have to make sure these two debouncers don't conflict to double the delay.