[HTTP/2.0] Server's API design feedback

[dhdavvie]

Hi all, for the past few weeks I have been working on developing an HTTP/2.0 server for wptserve(#11806 & #12193). I have started reaching a stage where a lot of the infrastructure is starting to be in place, and am looking to properly design and implement the API that test writers will be using in their tests.

I would love to hear your feedback regarding this, including possible use cases and in general what you would like to do/have access to. Currently there is a bit of support for tests in the H2ResponseWriter, for example you can push content using a handler like so:

    @wptserve.handlers.handler
    def handler(request, response):
        headers = [('server', 'test-h2')]
        response.writer.write_headers(headers, 202)

        promise_headers = [
            (':method', 'GET'),
            (':path', '/push-test'),
            (':scheme', 'https'),
            (':authority', '%s:%i' % (self.server.host, self.server.port))
        ]
        push_headers = [
            ('content-length', str(len(push_data))),
            ('content-type', 'text'),
        ]

        response.writer.write_push(
            promise_headers,
            push_stream_id=10,
            status=203,
            response_headers=push_headers,
            response_data=push_data
        )
        response.writer.write_data_frame(data, True)

I'm trying to increase the flexibility of the response writer, to give test writers more control over what the server does and in what order, please let me know what you think the response writer should allow you to do!

Another thing I would like to provide is the ability to make/manipulate frames, such as sending bogus frames as a response. The server currently stores the frames of a request in the request object, so there would also be the ability to see how the data was sent, to some extent. The hyper-h2 library simplifies some things, such as merging HEADER frames and any CONTINUATION frames into a single RequestReceived frame/event in their API for the end-user. I'm sure it would be possible to get access to the original frames as the hyper-h2 library received them, but currently that would not be a high priority item (or so I believe, let me know if you disagree).

I would really love to hear use cases, and what people want to be able to do with this server so that I can have a list of tasks that will help guide the design of the server. Please let me know if you want any clarifications :)

cc @jgraham @annevk @AutomatedTester @andreastt Please feel free to cc anyone who has any interest/stake in this

[jgraham]

So the model I had in mind was something like:

class ResponseHandler(SomeBaseClass):
    def handle_headers(self, request, response):
        pass

    def handle_data(self, request, response):
        pass

So as a user writing a .py test, instead of implementing a function called when the request is complete you would have the option to implement a class with one (option) method per frame type per incoming frame (after h2 normalisation). Access the Request object would give you access to all the frames recieved in this stream so far (although possibly the latest frame should go into the method signature). The response object would allow you to construct a response frame.

[annevk]

To restate some use cases for H/2 support on web-platform-tests from elsewhere:

• See how JavaScript APIs and various other client-side networking endpoints (e.g., <img>) behave with H/2. A concrete example here is testing xhr.statusText across implementations, which isn't interoperable at the moment.
• Test error handling in the client by sending bogus frames / bytes from the server. Ideally with the ability to do so at any point during the "transaction". (Also figure out if clients are indeed more restrictive byte-wise in what kind of values headers can contain and such.)

Maybe @mcmanus, @MattMenke2, or @bencebeky have further ideas here.

[annevk]

@wanderview and @jakearchibald you might have ideas too!

[LPardue]

Extension frame support is pretty important.

Using things like ALTSVC or ORIGIN leads on to an ability to test connection-oriented H2 features like connection coalescing.

Testing of extension points also provides a way to try greasing.

[LPardue]

Plus is this restricted the H2 or is HTTP/QUIC worth considering due to its similarities (many of the test cases can be reused even if syntax is a little different).

[dhdavvie]

@jgraham I really like that idea, I've implemented something like that which I will be pushing to the latest PR shortly.

@annevk I will take a look at the xhr stuff and see what can be done to create tests for that. I definitely agree with the bogus frames, and next on the agenda is working on that ability.

@LPardue That definitely looks very interesting, it might be useful at some point to have a way of defining custom frames and their handling, although currently I feel like that is a bit outside of the current scope as there is still a fair amount of base level stuff that needs to be in place. There is no plan to extend this to HTTP/QUIC, but if the similarities are great enough and the demand is there, I'm sure it would be possible in the future.

[domfarolino]

I would really love to hear use cases, and what people want to be able to do with this server so that I can have a list of tasks that will help guide the design of the server.

Worth documenting here that https://github.com/whatwg/fetch/pull/785 brings a good use-case of in some way being able to observe an H2 stream priority, so we can assert the H2 stream priorities of two different requests are the same.