Rework Handler/Request/Response Signatures

[lukeed]

Just brainstorming; nothing may actually come out of this!

I've been looking at the ServerRequest and ServerResponse interfaces a lot lately & wondering if they should even be here 🤔

Background

First, an overview of the PROs/CONs of everything, which lays out the reasoning for their existence:

ServerRequest

PROs

• the ServerRequest<P> TS interface makes for really nice DX as it can give confidence wrt req.params contents
• the req.body() "smart helper" is nice for auto-detecting & auto-parsing the body
• ...nice to still have direct access to the req.body.json() (and other) methods, if needed
• it's nice that new URL is done once and its values are stored on the request directly, so don't have to repeat

CONs

• See #52
  • Only specific properties are forwarded from Request to ServerRequest
  • Impossible to do req.clone as it no longer exists
  • Cannot support new Request(req.url, req) out of the box (related #52)
• It's invented, which means it requires setup/buy-in in order to use some (but not all) utilities Worktop offers
  • ws.connect doesn't have a preference
  • ws.listen returns a handler than needs to be given a ServerRequest because of the req.params usage within it
• A ServerRequest must be created from a FetchEvent in order for req.extend to exist
  • this is not ideal because Module Workers don't have FetchEvents

Visually

Compared to Request, these are the property differences:

  url: string;
++ path: string;
  method: Method;
++ origin: string;
++ hostname: string;
++ search: string;
++ query: URLSearchParams;
++ extend: FetchEvent['waitUntil'];
  cf: IncomingCloudflareProperties;
  headers: Headers;
++ params: P;
-- json(): Promise<any>;
-- formData(): Promise<FormData>;
-- arrayBuffer(): Promise<ArrayBuffer>;
-- blob(): Promise<Blob>;
-- text(): Promise<string>;
++ body: {
++  <T>(): Promise<T|void>;
++  json<T=any>(): Promise<T>;
++  arrayBuffer(): Promise<ArrayBuffer>;
++  formData(): Promise<FormData>;
++  text(): Promise<string>;
++  blob(): Promise<Blob>;
++ };
-- cache: RequestCache;
-- credentials: RequestCredentials;
-- destination: RequestDestination;
-- integrity: string;
-- keepalive: boolean;
-- mode: RequestMode;
-- redirect: RequestRedirect;
-- referrer: string;
-- referrerPolicy: ReferrerPolicy'
-- signal: AbortSignal;
-- clone(): Request;

ServerResponse

The main purpose of ServerResponse was to surface a Node.js-like API for composing responses.

PROs

• Has res.end, res.setHeader, res.writeHead and a bunch of others
• Still able to do new Response(res.body, res) for ServerResponse -> Response transform
• Contents remain mutable for request lifecycle (desired) until res.end or res.send is called
• Only relies on req.method for construction – used for HEAD checks
• Provides res.send for automatic Content-* headers and res.body serialization

CONs

• Custom, requires manual construction and/or adjustment for type match
• Does not satisfy native Response type requirements
  • AKA, can't pass a ServerResponse to an external library/helper if it wants a Response value
• Even though ServerResponse is tiny, it's still extra code

Handler

The Handler right now is strictly tied to the ServerRequest, ServerResponse pair. It makes sense for this to always have some worktop-specific signature to it, but the question boils down to whether or not ServerRequest and ServerResponse are the right base units.

The signature now is this:

type Handler<P> = (req: ServerRequest<P>, res: ServerResponse) => Promisable<Response|void>;

...which satisfies all "middleware" and "final/route handler" requirements. Worktop loops through all route handlers until either:

1. a Response or Promise<Response> is returned directly
2. the internal res.send or res.end have been called, which marks the ServerResponse as finished, and a Response is created from the ServerResponse contents

PROs

• Easily composable
• Satisfies middleware & final handler use cases
• Predictable & easily understood
• Allows mix/match of Promises
• No need for next()

CONs

• Requires ServerRequest and ServerResponse to be setup first

---

With the background out of the way, I have a few ideas of how these could be simplified and/or reworked to be compatible with libraries outside of worktop.

Important: All of these suggestions are breaking changes, which is not taken lightly.
If any of these are to happen, Worktop would have a strong division between old-vs-new through versioning – supporting the existing API and {new stuff} would not be considered.

Request Changes

1. Raw Request and add $ object for all customizations

This drops ServerRequest and instead uses Request directly, adding a $ property that has an object with all worktop extras. This would be like how Cloudflare adds the cf key with its own metadata.

interface RequestExtras<P extends Params> {
  params: P;
  path: string;
  origin: string;
  hostname: string;
  search: string;
  query: URLSearchParams;
  body<T>(): Promise<T | void>;
  extend: FetchEvent['waitUntil'];
}

interface Request<P extends Params = Params> {
  $: RequestExtras<P>;
}

TypeScript Playground

Breaks:

• req.params -> req.$.params
• req.path -> req.$.path
• req.* -> req.$.*
• req.body() -> req.$.body()
• req.body.json() -> req.json() (rely on native)
• req.body.*() -> req.*() (rely on native)
• req.extend() -> req.$.extend()

Additionally, I see two potential issues with this:

• It converts the Request interface to a generic, which means code can throw/cause TS errors when used outside of worktop. In other words, if someone tries to do Request<MyParams> outside of worktop, then that's gunna throw an error because Request, natively, is not a generic. Not having the generic means that worktop loses its params insights.
• Worktop can ensure req.$ always exists, but if a user writes a middleware/handler to be used outside of worktop, then something like req.$.path will throw a cannot access "path" of undefined error

2. Raw Request and use context object for all customizations

This is the exact same thing as above, but it uses the context key instead of $ for the object.

TypeScript Playground

3. Add all customizations directly to a Request object

Uses the incoming Request as is, but adds all the worktop customizations to it directly:

interface Request<P extends Params = Params> {
  params: P;
  path: string;
  origin: string;
  hostname: string;
  search: string;
  query: URLSearchParams;
  extend: FetchEvent['waitUntil'];
}

TypeScript Playground

Breaks:

• req.body() -> removed – use external utility
• req.body.json() -> removed – use req.json()
• req.body.*() -> removed – use req.*()

Additionally, this shares the same potential gotchas as Options 1 & 2

• Worktop assumes/requires Request to be a generic; might cause issues externally
• Worktop provides the req.params object; any user middleware running outside of worktop might include req.params.foo and req.params is not defined

Request Changes: Poll

---

Response Changes

There's not much that can really change about ServerResponse since it's all custom/worktop's to begin with... Really, there's only been on thing on my mind:

Move res.send to external utility

If res.send were exported as a top-level send utility (either from worktop/response or from worktop/utils), then people could use it externally.

In order to accomodate this change, send would have to return a Response directly. Right now it mutates the ServerResponse to signal the worktop.Router that the Handler loop is complete. All of the serialization & statusCode checks would happen within the send utility itself. However, it would be up to the Router to perform the HEAD check. This would keep its API more or less comparable to @polka/send.

Note: It's entirely possible to keep res.send and add a send export.

Response Changes: Poll

---

Handler Changes

As mentioned before, the Handler itself depends on decisions made to (Server)Request and (Server)Response. So any suggestions here will be made in addition to those, as this section is focusing on the Handler function signature itself.

Important: For brevity, I'll use Promise<Response> as a return type to refer to Promisable<Response | void>

1. Absolutely no changes

Keep the signature the same and use the exactly same ServerRequest and ServerResponse types:

type Handler<P> = (req: ServerRequest<P>, res: ServerResponse): Promise<Response>;

2. Keep the signature, but use Request type

Maintain the (req, res) parameters, but replace ServerRequest with one of the suggestions above.

type Handler<P> = (req: Request<P>, res: ServerResponse): Promise<Response>;

3. Use Request+ and a Context object

Note: Using Request+ to denote an overloaded/modified Request type; see above.

Changes the signature so that a Request is always used, leaving it up to Context to store additional/extra information. Your Handlers will be passing around the same Context object, which allows middleware/handlers (including Router.prepare) to mutate the context as needed.

For this Option 3 entry, the Context object looks like this:

type Context = {
  response: ServerResponse;
  waitUntil: FetchEvent['waitUntil'];
}

type Handler<
  P extends Params = Params,
  C extends Context = Context,
> = (req: Request<P>, context: C) => Promise<Response>;
// ^ this means you can inject your own `Context` types

4. Use pure Request and a Context object

Similar to Option 3, but moves all would-be ServerRequest extras into the Context object. This means that the Request is pure and has nothing added onto it (other than Cloudflare's cf property)

type Context<P extends Params = Params> = {
  // Request extras
  params: P;
  path: string;
  origin: string;
  hostname: string;
  search: string;
  query: URLSearchParams;
  // the FetchEvent information
  extend: FetchEvent['waitUntil']; // name TBD
  waitUntil: FetchEvent['waitUntil']; // name TBD
  passThroughOnException: FetchEvent['passThroughOnException']; // name TBD
  // the ServerResponse 
  response: ServerResponse;
}

type Handler<
  P extends Params = Params,
  C extends Context = Context,
> = (req: Request, context: C<P>) => Promise<Response>;
// ^ this means you can inject your own `Context` types

5. Only receive a Context parameter

Change the Handler signature so that it's just receiving a single object with everything in it. It may look something like this:

interface Context<P extends Params = Params> {
  // raw request
  request: Request;
  // the ServerResponse 
  response: ServerResponse;

  // request extras
  params: P;
  path: string;
  origin: string;
  hostname: string;
  search: string;
  query: URLSearchParams;

  // the FetchEvent information
  extend: FetchEvent['waitUntil']; // name TBD
  waitUntil: FetchEvent['waitUntil']; // name TBD
  passThroughOnException: FetchEvent['passThroughOnException']; // name TBD
}

type Handler<
  P extends Params = Params,
  C extends Context = Context,
> = (context: C<P>) => Promise<Response>;

Handler Changes: Poll

Now, if you voted for something that involved a Context object, should worktop auto-create a ServerResponse for you?

All Context-based answers assume that ServerRequest is gone, shifting its properties either to the req directly or to the Context object. In this world, Worktop can continue providing a context.response (name TBD) for you, or you can create it yourself as part of your Router.prepare hook.

The purists among you may have despised the wannabe-Nodejs all this time, so this would be the opportunity to explicitly opt into ServerResponse only when needed ... something like this:

import { Router } from 'worktop';
import { ServerResponse } from 'worktop/response';
import type { Context } from 'worktop';

interface MyContext extends Context {
  response: ServerResponse;
}

const API = new Router<MyContext>();

API.prepare = function (req, context) {
  // assumes no changes to ServerResponse API
  context.response = new ServerResponse(req.method);
}
// or
API.prepare = function (context) {
  // assumes no changes to ServerResponse API
  context.response = new ServerResponse(context.request.method);
}

---

Thank you!

I know this a lot (too much) to read and sift through. If you managed to go through it – or even some of it – thank you so much. I really really appreciate the feedback.

[lukeed]

Ok, so I was having a play in the TS playground a bit, combing some of the ideas in here:

• Change the Handler signature to accept two params: a raw Request and a Context object
• Remove ServerRequest entirely:
  • moves req.params to context.params
  • moves req.search (and others) to context.url.search (and others)
    In fact, context.url is the raw new URL(req.url) object, so you can reference it wherever/whenever.
  • moved req.query -> context.url.searchParams
    I might add a context.url.query alias.
  • moves the req.body() smart helper to a worktop/utils export.
• Does not auto-construct ServerResponse for you
  In this example, "user" chooses to construct it as context.res during the API.prepare hook.
• Changes ServerResponse.send and ServerResponse.end so that they return Response directly

View in TypeScript Playground

Note: You'll have to scroll towards the bottom to see user-facing code!

[lukeed]

Closing this as it – well, some form of "it" – has been implemented in #83

This is going to be available as worktop@next and I will continue to experiment with the new system under these @next tags. Anyone is welcome to try it out too, of course!

Once I have enough confidence in the new arrangement, this work will be promoted to the main/latest worktop stable release.