pimterry
commented
1 year ago Yes that would definitely be good! This relates to gRPC support, but it would also be useful elsewhere.
The way this works currently is that Mockttp records all response data (here) waits until the whole response has been sent back to the client (status code, headers & body) and then fires a single event with that data (here), but obviously that doesn't work very well if the response stays open and slowly streams data for a long time though (and with gRPC, I think it can even be streaming in both directions, to make this extra complicated).
This is solvable, but we need to decide how... I guess maybe we start an interval (maybe running every 1 second?) after we start sending the request & response bodies, and if we haven't finished yet, we fire an event with the latest partial body data? That could work, and should be easy enough to handle and show in the UI afterwards. Updating every second should make data appear fast enough for most use cases. Or we could do a cleverer debouncing kind of approach: fire partial data almost immediately, then fire throttled events, so that events for new data are usually sent immediately on relatively quiet connections (<1 new chunk of data per second), but very frequent updates (>1 new chunk per second) are batched up for efficiency.
Notably, for gRPC streaming calls it's possible that request & response bodies can be going both ways at the same time - this is bidirectional streaming rpc. We should make sure it's possible to handle that case, which means firing request & response events with enough information that you can work out the overall order of data sent and received (so maybe every event is a series of chunks where each one has a timestamp, so we can work out the detailed timing given both request & response messages).
Compression needs some thought - in theory we can decompress a partially received body that's gzipped, but I've never tried it and there's probably some details to figure out there (or we just ignore it for now, and we never stream compressed data).
Anyway, that's just some notes for implementing this later. I'm not going to have time to do this immediately (currently busy adding interception for WebRTC) but if you or anybody else is especially interested then you're welcome to start work on it! The first step would be creating a response-initiated event in Mockttp, which fires when the status & headers have been sent, but before the body is sent, just like the existing request-initiated event. Then we'd need to extend this further to test some approaches to body streaming events, probably announcing the latest body data at frequent intervals as above. Just adding that first event would be a useful small change though. Once all the Mockttp parts are in to make the data available, the UI work should be easy.
Firebase etc. have streaming responses.
https://firestore.googleapis.com/google.firestore.v1.Firestore/Listen/channelIt would be nice if HttpToolKit can display it.