[💡FEATURE REQUEST]: Streaming request [WIP] & responses [HTTP-DONE]

[lwc]

This project looks like it could be very useful in building a GRPC server bridge for PHP, which would be amazing.

GRPC has the concept of streaming request & responses, is that something you'd consider adding to roadrunner/goridge?

Thanks

---

This ticket (since most of the users know it) split into the four parts:

• [x] HTTP response streaming [done in v2023.3]
• [ ] HTTP request streaming
• [ ] gRPC response streaming
• [ ] gRPC request streaming

[wolfy-j]

It can be done by locking done and working with one worker within the request. This is definitely something we are going to consider adding going forward.

[wolfy-j]

I have put a foundation for streaming responses into a newer version. Streaming requests will be looked at once this feature it complete. See spiral/roadrunner#9

[wolfy-j]

This will be coming in a separate server due async mode of underlying workers.

[pwilsoncentre]

I would love to see streaming support as well, but for large HTTP responses. I noticed this issue was reopened a few days ago, so does that mean that some progress will be made here? If I had time to learn Go I'd offer to help, but as it is, I can only offer my encouragement - this would be a really cool feature to have.

[wolfy-j]

Yes, this issue will be reconsidered for 1.5.0. No timeline yet but there are some initial experiments already made.

[wolfy-j]

Not in 1.5.0. We are working on a protocol update (with few extra features and perf improvements), so most likely it will be 2.0.0.

[Ocramius]

Ha, this is quite a roadblock: I was trying to migrate an existing system to RR this weekend, and one of the payloads that we stream is a GIGANTIC XML Sitemap that needs to be produced chunk-by-chunk (especially important to avoid running into memory limits).

I'm unsure about whether this is to be done in the GO components, or in the PHP-side of things (PSR-7 stream response missing there too, perhaps?).

I haven't done a detailed analysis, but this is indeed the biggest limitation I faced so far with the tool, whereas everything else is, I must say, quite impressive :-)

[wolfy-j]

With RR2 refactor we might be able to take a look at this issue with a fresh approach. We will discuss this internally to see if it is possible.

[rustatian]

Discussion: Local pipelining with jobs + broadcast plugin

[wolfy-j]

Essentially we have an idea of how to solve this issue with: 1) No memory leaks 2) No blocking on HTTP worker 3) Proper load balancing and prevention of over saturation 4) Side effect - extra feature reliable long polling in PHP 5) Horizontal scalability

[wolfy-j]

@rustatian we should consider doing the same for GRPC (streaming).

[rustatian]

A little update about streaming file responses. The sendfile HTTP middleware handles the X-Sendfile header and streams the response back w/o involving the worker in the process. So, you are able to handle the requests with the PHP worker and stream the file(s) in parallel. More info here: https://github.com/spiral/roadrunner-plugins/issues/9, PR https://github.com/spiral/roadrunner-plugins/pull/56.

[Ocramius]

@rustatian streaming files usually not the problem though: streaming batches (think "massive CSV generated from DB") is still the issue though

[rustatian]

@rustatian streaming files usually not the problem though: streaming batches (think "massive CSV generated from DB") is still the issue though

Yep, sure. That's was just a little update about streaming in general. Not about the ticket's topic.

[WagnerPMC]

+1 It would be nice to see a ready implementation of this feature request

[rustatian]

+1

It would be nice to see a ready implementation of this feature request

No +1 please. If you want to support the ticket, please, like the issue itself.

[rustatian]

Update: we've started working on the streaming responses. Expected to be in the RRv2.9.0 (according to our draft plan).

[rustatian]

Ok, guys, I can confirm, that v2.9 will bring us streaming responses.

You'll be able to turn on streaming capabilities in the experimental (new) section.

Later, we will support the metadata, headers, and other grpc-streaming like features.

[rustatian]

Ok, guys, the time has come ⌛ https://github.com/roadrunner-server/roadrunner/releases/tag/v2.9.0-alpha.1 Please, read the release notes very carefully. Put your feedback in the discussion (link in the release notes). Feel free to join our discord server to ask your questions. We are always happy to help. Link: https://discord.gg/TFeEmCs

[Ocramius]

Unable to test for now (moved a project where I wanted to use rr to ext-openswoole, as a workaround) but very promising to finally have this 💪

[rustatian]

Unable to test for now (moved a project where I wanted to use rr to ext-openswoole, as a workaround) but very promising to finally have this muscle

Sure, np 👍🏻. Just curious, what were the points to choosing swoole over the rr ? Streaming?

[rustatian]

The response streaming feature will be in the 2.10-alpha releases because we should appropriately test it. We are still waiting for the feedback 😜 In the next release, we plan to update a PHP worker (https://github.com/spiral/goridge-php/issues/12) to have bi-directional communication.

[rustatian]

Hey, community 👋🏻 Response streaming will be in the v2.13.0 version by default (supported by both, RR and PHP workers).

[tux-rampage]

Thanks for all the effort you guys are putting into this.

How should this work? Can the php worker delegate the response sending to another process (maybe a php job, a go service, nodejs service, etc...) which will then stream the response and complete the request via roadrunners RPC or am I on the wrong track?

Could you briefly outline how this should work?

[rustatian]

Yeah, sure.

1. We have two plugins that would support this feature -> http and gRPC.

For the http plugin, we will support the response streaming. We did this previously in the v2.10, but I decided to not include that because of some known bugs which may affect the users.

For the gRPC plugin, both response and request streaming would be supported. So basically, that would block the worker during the stream, but you would have the ability to stream some big payloads from some external system, for example, and handle it via your PHP worker.

Before the release, I'll release the usual beta-rc binaries, so, anyone might test that and put the feedback here. So, I would be able to tune this feature before the release.

I can't explain how this would work from the PHP side 😭, you may ask our PHP team in the Discord 😃

[Matt-Woz]

Hi, are there any updates on how this is going? Do we have an ETA on when this might be released? Thanks

[rustatian]

Hey @Matt-Woz 👋🏻 PHP worker and php-goridge transport have recently been updated to support response streaming. RR (Golang) part is planned for the v2023.3.0.

[FluffyDiscord]

Would it be possible to add also the HTTP request streaming after the move of internal communication from JSON to protobuf? It would make big file uploads extremely simple to do, instead of the current approach of sending small chunks. (Is there currently a better way?)

[rustatian]

@FluffyDiscord Sure 😃 If someone willing to help me (with the PHP part) with that, I'm all ears 👂 Since I'm not a PHP dev at all.

[FluffyDiscord]

I think RR could create and pass to PHP a port (assuming locally running RR and PHP app) or TCP endpoint alongside the normal request. We could then read from the opened socket. Basically RR would be just a passthrough. Then whenever PHP returns a response the stream would close.

I am completely new in communication using a sockets, but I think this is the only way. Saving the request as temp file (not talking about sockets) would be probably an option too, but I would like to not use file buffering.

This might be more of a question to the Spiral team, which are experienced with this kind of stuff I imagine.

[L3tum]

I don't think that's even necessary. I'm not sure how protobuf comms work, but since the worker comms are running in a goroutine in RR, blocking on sending the request shouldn't pose a problem, so in PHP we simply would need to signal that the request is ongoing. Thinking about it, RR could enable it by sending a request in 2-3 frames (signal for stream + request line, headers, body), with the second/third frame only after it was requested by the PHP side. In return the PHP side would not need much change. Nyholm's PSR7 request object already supports a lazily loaded body, so there should be minimal change necessary (most likely the framework integrations would need to add support for wrapping the RPC request for the request body in some manner for the framework). I'm not sure what work would need to be done in Go though, since I don't exactly know if it's waiting for the full request right now or already streams it internally. But if you need some support for this @rustatian hit me up (after the async + circuitbreaker stuff 😆 ) :)