WebSocket support

[zostay]

I am just starting my research on this topic. I've been looking at uWSGI and Mojolicious to see how they work and both operate on a more reactive basis, one we can totally implement in Perl6. I think we might be able to duplicate this idea with a P6SGI application with something like this:

sub app(%env) {
    start {
        my $ws-events = %env<p6sgix.websocket>.handshake(...); # some stuff in here
        200, [], on -> $res {
            $ws-events => -> $event {
                ...; # process the event
                $res.emit($response);
            },
        }
    }
}

It might be as simple as that. I need to do more research, but I'm sure we can come up with something sensible.

[muraiki]

I'm still reading through the P6SGI spec alongside learning Perl 6, so I apologize if my comments aren't the most helpful.

When considering websocket usage we should also think about pushing events from the server to one or more clients, in addition to responding reactively to events from a single client. For instance, say that we want to emit an event to a client whenever a new file shows up in a directory. My concern is that in the websocket context does it even make sense to return the response promise? And what do we return as the Supply if we want to emit events from the server to multiople clients? I guess that I'm not sure how to design something that lets people compose middleware across both the HTTP req/response paradigm and websockets, but I might be missing part of the picture here.

Perhaps one experiment would be to somehow translate websocket events into "normal" events, and then fit HTTP's request/response cycle into that framework -- vs trying to fit ws into the p6sgi-like framework. In this system, you'd basically be defining a graph of connected supplies, and middleware becomes something you insert into that graph. I'm thinking of something more along the lines of Cycle.js which is based upon Reactive Extensions (P6 supplies are essentially an implementation of Rx), but on the backend.

I think there's merit in thinking about structuring web apps as connecting sources and receivers of events (unifying both WS and HTTP) vs a more one-directional approach of a pipeline of HTTP-oriented middleware that also has this WS code added on top. Does the latter case exist because it is ideal or is it a byproduct of the history of the evolution of the web in languages that don't have the best async/concurrent support?

All this is to say that the types of applications that use WS could differ greatly in structure from what P6SGI/uWSGI/etc. implicitly bind you to. Maybe I'm just rambling nonsense. I'll most likely be at the Pittsburgh Perl Workshop so perhaps I can ramble to you in person there. :)

For something less radical you might want to look into WS implementations in languages that have good async/concurrency support built in, like Phoenix for Elixir and the various WS + Akka approaches for Scala.

[zostay]

While I know Perl 6 and P6SGI, my comments on the subject are off because I'm learning WS. I kind of know it's a like an MQ and some MQs implement WS, but really haven't made as much use of such things as I'd like to. You are raising good points.

Regarding the Promise, I'm starting sour on them being part of the application (see #13) as perhaps it's best to leave the details of managing the main worker thread to the server. The usual way of creating a Promise, via a start-block, inherently throws that work into a separate thread controlled by the app. There's nothing very wrong with that, but maybe it is just better to make the main bit of the app continue in the main thread unless it needs to do something fancy.

Before we finalize anything, we really need Smack to implement WS so we can toy around with the consequences in some tests and see what works and what doesn't. A variety of use cases would be good, so I'll start studying WS and see what we can do on that side of things.

Thanks for those references. I've dabbled in Scala, but hadn't realized Akka had WS bindings. I should look there too. I really think Scala is very Perl 6-ish, but with an unfortunate Java-y aftertaste.

Rambling is not a bad thing when you're first brainstorming and trying to come up with something new, so feel free to ramble.

[zostay]

Actually, I'm reading RFC 6455 and my initial thoughts are that perhaps what we just need is to provide p6sgi.input as a Supply. Then, an application implementing WebSocket could look like this (assuming we're keeping the Promise in the app, which could change):

sub app(%env) {
    start {
        if %env<HTTP_UPGRADE> eq 'websocket' {
            101, [
                Upgrade => 'websocket',
                Connection => 'Upgrade',
                Sec-WebSocket-Accept => process-handshake(%env),
            ], on -> $res {
                %env<p6sgi.input> => -> $m {
                    process-frame($m, $res);
                },
            };
        }
        else {
            400, [ Content-type => 'text/plain' ], [ 'Bad Request' ]
        }
    }
}

The WS can emit frames to the response whenever it wants to and process the incoming frames from the input whenever they arrive. When finished, it emits the close control frame and then tells the response Supply that it's done.

Anyway, based on my still limited understanding of WS, this looks pretty reasonable.

[zostay]

I think HTTP/2 could be implemented this way too, which suggests to me that this is too low level. Protocol details really ought to be handled by the server. More thought is needed.

[zostay]

So, an application being able to do the above is not wrong, but it is more complex than I really want an application to deal with. I'm thinking of adding special Protocol section to P6SGI, which would typically be implemented by the server (but might be implemented by middleware, so long as the server is willing to hold the socket open).

Instead of the above, we might do something like this instead:

# WebSocket echo server with a counter every second
sub app(%env) {
    start {
        my Supply $interval .= interval(1);
        if %env<HTTP_UPGRADE> eq 'websocket' {
            101, [
                X-P6SGI-Upgrade => 'websocket', # notify server to perform an upgrade to websocket from HTTP/1
            ], on -> $res {
                %env<p6sgi.input> => -> %m {
                    if %m<p6sgix.ws.opcode> eq 0x1 { # text
                       %env<p6sgi.errors>.emit(%m<p6sgix.ws.data>);
                        $res.emit({ 'p6sgix.ws.opcode' => 0x1, 'p6sgix.ws.data' => %m<p6sgix.ws.data> });
                    }
                },
                $interval => -> $count {
                    $res.emit({ 'p6sgix.ws.opcode' => 0x1, 'p6sgix.ws.data' => $count });
                },
            };
        }
        else {
            400, [ Content-type => 'text/plain' ], [ 'Bad Request' ]
        }
    }
}

This allows for other protocol extensions to be added in the future. At least, that's one idea.

[zostay]

Honestly, though, sending DATA frames as text or binary probably ought to be handled over a WebSocket connection by emitted Str and Blob, like it works over the HTTP connection prior to Upgrade. I also still don't like how the Upgrade is handled in the above example.

[zostay]

I have added the Protocol section to mention how WebSocket must be handled by the server. If the server initiates a WebSocket on its own, it will pass the SERVER_PROTOCOL in as "WebSocket/13" and the input/output streams are used to send frames continuously over the connection. The server is responsible for decoding and pulling data out of the frames so the application just has to worry about the details of it's own protocol.

For WebSocket, however, it will probably be more normal for the application to perform its own Upgrade. For this, I have provided the Protocol Upgrade extension, which allows the application to direct the server to upgrade on request.

I'm calling this done, though I'm sure it will need more tweaking.