Rack 2 should be HTTP/2 friendly

[igrigorik]

Current proposal hardcodes HTTP/1.1 semantics: header processing and encoding, serialized request processing, lack of flow control, etc. If the goal is to make "HTTP landscape for Ruby amazing", we should instead aim to deliver an awesome HTTP/2 friendly rack implementation... and backport HTTP/1.1 where possible.

FWIW, my first run at a pure Ruby implementation of HTTP/2 protocol: https://github.com/igrigorik/http-2 - note that its based on draft-6, which is about 8 drafts behind latest.. but I'm hoping to close that gap soon(ish). I'm sure the API (and perf) could use a lot of love and attention, but my goal was to make it work first.

P.S. As far as timelines go, HTTP/2 is in last call, and with luck we'll have a stable spec by end of year.

[tenderlove]

Current proposal hardcodes HTTP/1.1 semantics: header processing and encoding, serialized request processing, lack of flow control, etc.

Can you be more specific? The point of provided request / response objects is that the server can provide custom instances with whatever extensions (like HTTP/2) they want.

and backport HTTP/1.1 where possible.

This is basically a non starter. We have tons of existing 1.1 apps and webservers that need something better than we have today. I want something that works well with 1.1 but is upgradable to 2. Do you have any suggestions for that?

[tenderlove]

To make it clear, I do want to be HTTP2 friendly, but I don't want to drop all the existing Ruby webservers. I'd really appreciate help developing an API that can work with both (not supporting HTTP2 feature on HTTP1.1 webservers, but an API that can upgrade). :-)

[allyraza]

IMO we should let the server handle that, this could cause a lot of problems. If puma team wants to implement HTTP/2 they go ahead and implement it does not effect anybody.

[igrigorik]

@tenderlove we need to decouple request parsing and serialization from rack, or push it down a layer - i.e. the app API shouldn't assume that it has direct read or write access to a socket. Instead, it should pass down the headers and the response stream to a lower layer that can then determine how the request should be encoded, and vice versa, how inbound bytes should be decoded before the response is passed up to the app.

Case in point, same server can be servicing HTTP/1.1 and HTTP/2 connections, and the encoding rules will vary by socket. Also, note that header encoding in HTTP/2 must be done at a lower layer (not on a per request basis) due to how the compressor is implemented: http://http2.github.io/http2-spec/compression.html... With HTTP/2, a single request also doesn't "hijack" the socket. Instead, multiple requests and responses can be reading and writing from it. Finally, there are also questions about handling priorities, flow control, server push.

I don't have a concrete proposal for how this API should look, but just wanted to raise the flag early on.

[mcary]

Other protocols that might fall somewhere between HTTP 1.x and HTTP 2 are:

• Websockets - can a server support upgrading to websockets with the current request/response API you've already proposed?
• SSE - I think this is already supportable with the new flexibility of writing to the response socket
• Large upload/download - OK, this is strictly speaking just HTTP 1.x in terms of protocol, but can a web server conforming to the request/response API you've outlined do it efficiently? How would "sendfile" support work? Or something similar for uploads; a mechanism that doesn't require a Ruby thread or perhaps even any user-space time to receive a file.

One idea is to have a bridge between a connection-oriented protocol and the request/response oriented API.

class Listener
  def call connection
  end
end

class RequestListener < Listener
  def call connection
    # ... create req and res from connection ...
    RequestHandler.new.call req, res
    # ... write res back to connection if that hasn't already been done ...
  end
end

(RequestHandler corresponds roughly to Application in the current request/response concept.)

I'm thinking that an HTTP2-compatible RequestListener might be doable if res doesn't write directly to connection or somehow mediates the output with other res objects that are attached to that connection.

Another option would be to unify the two:

class Application
  def initialize connection
  end

  def call req, res
  end
end

But I'd rather have two separate smaller interfaces/protocols.

I don't think this addresses the efficient upload/download scenario very well, but maybe it covers the others?

[mcary]

Here are some further thoughts about HTTP 2 and the other use cases that I saw as being related:

• Websockets
• Sendfile
• Uploads
• HTTP 2 (Push and Priority)

Websockets

Reading up on websockets, I think the handshake can be done well enough on top of the current API, because a websockets connection works very much like a single HTTP request/response with the exception that each party continues appending to the HTTP body it sent and reading from the one it received indefinitely.

I'm not sure if Rack2 should include a websockets API, but if it did, it might look like this, in Application#call:

  upgraded = res.websockets_upgrade do |in, out|
    out.send_frame "Hello World"
    frame = in.recieve_frame
  end

Ideally, a webserver could be told to release application resources on that connection, such as the thread and application context, without closing the connection. For example, websockets are often used to push updates to the client, in which case the application would want to collect the out streams in an array somewhere and feed data to them later, possibly from a background thread or while processing a request from a different connection. Maybe that would look like this:

  CONNECTIONS.push out.leave_open!

By the way, I needed a way to access the headers of the request in order to implement websocket upgrades, and I don't think the current Rack2 proposal has one. How about:

  header_value = req.header("Sec-WebSocket-Key")

I like this approach better than explosing a req.headers hash because it allows for normalizing header names. I like this approach better than custom methods for common header names (req.user_agent) because it allows for a single consistent API for all headers. The one thing I don't like is that it doesn't raise an error if I mistype the header name, the way a custom method would. But if it raise on missing/misspelled headers, then I have to check for presence every time I access it.

Sendfile

Rails already has some support for sendfile (See Linux or BSD man pages) via the Rails send_file method.

The idea is to efficiently copy bytes to a socket when serving from the filesystem, for example when serving large assets. Once the application determines which file to send, the server can release application resources such as a thread and application context, and copy the bytes as efficiently as possible. On Linux, some servers (Apache) take advantage of the sendfile system call which handles all the work in the kernel for maximum efficiency. Some configurations also automatically take care of cache validation headers like Etag and If-Modified-Since automatically.

This can already be addressed on Apache by setting the 'X-Sendfile' header, but that API for this feature is a bit limiting. For example, the sendfile system call also allows specifying an offset and length, which would be useful with a Range header to recover from a partial transfer. I suppose we could settle for X-Sendfile: path; offset=1024; length=1024, but then, we could also get by with Rack 1.x I/O hijacking, but we are looking for a better solution, aren't we?

So maybe:

  res.sendfile(path, offset: x, length: y)

I'd love to see Ruby webservers able to support this kind of feature, including Unicorn and Puma, and without Apache. Even if a sendfile is not available, the server could offload the byte copying to an evented reactor or something.

Uploads

I'd also love to see better support for uploads that work similarly to sendfile: support resumable transfer, avoid tying up application threads, etc. One idea is to invoke the application on a thread as usual, and let it supply a block for handling the remainder of the request later, when the upload is fully received. That gives the application a chance to apply access control and Content-Length policies before proceeding.

  req.receive_file(local_filename) do
    # process the uploaded file later
  end
  # exit immediately

Since this is probably not going to be handled by the kernel, it might be more feasible to have some sort of status callbacks to be called periodically.

  cb = proc {|bytes_received| ... }
  req.receive_file(local_filename, status: cb, status_interval: 5.seconds) do
    # process the uploaded file
  end
  # exit immediately

For multi-part uploads, ideally we'd be able to invoke this on just the current part, and continue processing the remaining parts inside the block.

An existing solution for uploads is mod_porter (https://github.com/actionrails/modporter) but it requires Apache, and I'd like to see Puma and Unicorn supporting file uploads efficiently without relying on Apache.

HTTP 2 Push

The HTTP 2 feature I'd most want to take advantage of is "server push" aka "push promise" aka responding with additional resources.

I haven't come up with a compelling reason to expose the connection, which transcends an individual request/response cycle, as mentioned in my previous comment.

Server push is supported today with Apache's mod_spdy by setting a custom header: X-Associated-Content (http://chimera.labs.oreilly.com/books/1230000000545/ch12.html#IMPLEMENTING_PUSH)

However, HTTP 2 needs a set of headers to attribute to the additional resource, and it might be good to let the application control that (for example, by setting 'Accept: text/css' for stylesheets). That would be hard to encode in an X-Associated Content header. An API might look like:

  res.push_promise(new_req, priority)

This would cause the webserver to send a push_promise frame, and immediately start or queue processing for the new_req. The new_req is processed in the usual way (via Application#call). Depending on application processing time for the resources, HTTP 2 might even allow the promised resource to be sent to the client first so that critical CSS is going down the pipe while the application is still thinking about the base HTML.

While I like the idea of giving the application full control over the new_req, it's probably more cumbersome than the common case of just naming the promised URI. Maybe headers would be better specified in an optional hash?

  headers = { accept: 'text/css' }
  res.push_promise(new_uri, priority, headers)

I'd like to see this manifested in a Rails feature along the lines of:

  <%= stylesheet_tag_or_inline 'above_the_fold' %>

This would automatically send a push_promise of using HTTP 2 with push_promise enabled, or just inline the CSS if push_promise is not available.

I think @tenderlove's goal is to stick to APIs that are supportable on current servers like Puma/Unicorn, even before they get HTTP 2 support. I don't see a way to support this feature in that case, but actually, the client could disable push_promise in HTTP 2 or just be using HTTP 1.1, which means the application would need to be able to check for push_promise availability anyway. For servers that don't support it, that check could just always return false.

  res.push_promise_enabled?

HTTP 2 Priority

Another HTTP 2 feature that would be useful to expose to the application is stream prioritization.

In HTTP 2, prioritization is per-connection. A client is expected to establish a single connection to download an entire web page instead of the usual ~6 for HTTP 1.1. Each resource is requested within its own stream, many of which are multiplexed over the single single TCP/TLS connection.

The application may be able to help determine appropriate resource allocation based on the priority of the stream, but the ways in which resource allocations can be tuned depends on the server's capabilities. I'm not sure where to take that. Maybe for starts we just allow the application to know the stream priority and connection ID?

Examples of ways a server could back-burner a low-priority request:

• Using Unix "nice" to reduce the dynamic priority of the process in a process-based concurrency model
• Setting a "batch" thread scheduling policy for pthreads... not sure how prone this is to starving the thread, but might be appropriate for large upload/download
• With push_promise, waiting till after the current request to queue the promised one
• Using a different database slave to service the request
• Limiting bandwidth of a stream until other ones have finished

There are no absolute priority ranges, so how a server handles a particular request depends on the priority of others in progress for the connection, which can change during the execution of the request.

In addition to letting prioritization deal with contention between multiple client requests on a connection, it would be great to let prioritization deal with the contention between promised resources and the original request that triggered the promise.

And I suppose it could be useful to let the server set priorities for clients that don't, for example ones using HTTP 1.1, based on content type. For example, it might be useful to encourage base HTML and CSS to be transferred at a higher priority than images or large files.

Next Steps

So what do other folks think of these ideas? Is there interest? Does it make sense to include them in Rack 2 vs. an add-on or extension to that specification?

[tenderlove]

@igrigorik

we need to decouple request parsing and serialization from rack, or push it down a layer - i.e. the app API shouldn't assume that it has direct read or write access to a socket. Instead, it should pass down the headers and the response stream to a lower layer that can then determine how the request should be encoded, and vice versa, how inbound bytes should be decoded before the response is passed up to the app.

This is why we give applications a request and response object. These can be backed by anything. Inbound and outbound encoding is determined by the request and response object.

Case in point, same server can be servicing HTTP/1.1 and HTTP/2 connections, and the encoding rules will vary by socket.

Right, and the request and response object type given to the app should change depending on the version of HTTP we're servicing.

@mcary I like where you're going. The proposal seems viable and I think we can integrate what you're suggesting in to the request and response objects over time.

Does it make sense to include them in Rack 2 vs. an add-on or extension to that specification?

I prefer we make it an extension for now. The only reason I say that is that I want a low barrier to entry for existing web servers. However, we should roll that API up to the main "spec" over time.

My major point with this is that we should use objects that we can extend to upgrade to HTTP2, and I think your proposal does that.

[igrigorik]

I'm thinking that an HTTP2-compatible RequestListener might be doable if res doesn't write directly to connection or somehow mediates the output with other res objects that are attached to that connection.

Yes, I think that's a step in the right direction. Note that this is not HTTP/2 only either. If you want to support HTTP/1.1 pipelining then you also need to decouple connection and request objects.

Re, sendfile, uploads: this stuff gets tricky. Note that sendfile can't simply pump bytes into the open socket, as it does now, with HTTP/2 it'll have to frame those bytes, respect flow control, etc. Similarly, for uploads, you also need to support streaming uploads -- buffered uploads should be an option, but not the default.

Re, server push: I think there is some misunderstanding here. You don't set additional request headers, instead the push response inherits original request headers, modulo resource path.

See server push example here: https://github.com/igrigorik/http-2#server-push

HTTP 2 might even allow the promised resource to be sent to the client first so that critical CSS is going down the pipe while the application is still thinking about the base HTML

To clarify, promise headers must be sent before request response body. After that, data can be interleaved in any order. Also, there is prioritization and flow control:

• https://github.com/igrigorik/http-2#prioritization
• https://github.com/igrigorik/http-2#flow-control

---

tl;dr: decoupling connection object from req/res is the right and necessary first step. From there we can iterate how to expose the right APIs on connection + req/res objects.

P.S. Example of http/2 server: https://github.com/igrigorik/http-2/blob/master/example/server.rb#L32

[lozandier]

Is there an update on this? There doesn't seem to be anything esily findable on the Web or Google about Rack 2 & HTTP/2 beyond this discussion & https://github.com/Wardrop/Rack-Next

[stakach]

Sorry if this is a bit off topic, I do think it is somewhat relevant to the conversation.

I've been working on a Ruby webserver and I'm planning on using @igrigorik HTTP2 implementation as part of the parsing layer.

Just for a quick overview of where it is at:

• Can launch multiple ruby applications simultaneously
• Can run these applications in process / cluster mode or threaded mode
  • Both modes can run simultaneously although each application is pegged to a mode
• Supports non-blocking vectored IO for both file system and network
  • So should be able to send files of any size efficiently over HTTP2
• Performs Application-Layer Protocol Negotiation for HTTP2 support when using TLS
  • Defaults to HTTP1.1 otherwise

My question effectively pertains to websockets and how I should be implementing these with HTTP2. Or should these remain a HTTP1.1 upgrade and run over a dedicated TCP connection? I'm currently using https://github.com/faye/websocket-driver-ruby for upgraded HTTP1.1 however I'm not sure how this translates when using HTTP2...

On a side note, I like ideas presented by @mcary and once again, sorry if this off topic

[igrigorik]

My question effectively pertains to websockets and how I should be implementing these with HTTP2. Or should these remain a HTTP1.1 upgrade and run over a dedicated TCP connection? I'm currently using https://github.com/faye/websocket-driver-ruby for upgraded HTTP1.1 however I'm not sure how this translates when using HTTP2...

It doesn't. There have been a few discussions about this (how to map WS to h2), but those haven't resulted in anything concrete. As of today, there is no "WS over H2" mapping... And by the looks of it, probably won't be: h2 removes many of the reasons why WS was introduced in the first place.

[stakach]

Seems reasonable. So chrome, for example, would open a new connection for a websocket.

Out of interest how does one replicate websocket functionality over H2? Is it possible with evergreen browsers today?

That way one could abstract application logic from protocol and have a JS client select the best available transport.

[igrigorik]

Seems reasonable. So chrome, for example, would open a new connection for a websocket.

Correct. It falls back to existing HTTP/1.1 upgrade flow.

Out of interest how does one replicate websocket functionality over H2?

Each message is a stream: client can open as many as it wants; server can use server push or respond to client's 'hanging GET's'. There are multiple ways to skin it.. The server push route is probably the best one for server-initiated messages, but one thing that's lacking is a JS API to get callbacks on pushed streams (WIP).

[boazsegev]

HTTP/2 Server Push isn't reliable and might be intercepted and cut-off by intermediaries. From the current draft:

An intermediary can receive pushes from the server and choose not to forward them on to the client. In other words, how to make use of the pushed information is up to that intermediary. Equally, the intermediary might choose to make additional pushes to the client, without any action taken by the server.

Hence, Websocket connections are more reliable for pushing data than Push-Promise...

It seems that the Websocket negotiation will move to the ALPN when using TLS. Otherwise, when using clear-text, the HTTP/1.1 upgrade will probably stay as it is (HTTP/2 over clear text uses the same upgrade mechanism and there is no reason to move through the same mechanism twice).

[igrigorik]

HTTP/2 Server Push isn't reliable and might be intercepted and cut-off by intermediaries.

Note that browsers are implementing h2 over TLS only, which means you have explicit knowledge and control over your intermediaries. As such, you can ensure that push is forwarded.