Doesn't support auto-coercion for response bodies

tmcw commented 5 months ago

Prerequisites

[X] I have written a descriptive issue title
[X] I have searched existing issues to ensure the bug has not already been reported

Fastify version

4.26.1

Plugin version

4.0.0

Node.js version

v20.11.0

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

14.3 (23D56)

Description

I filed an issue and submitted a PR over on fast-json-stringify https://github.com/fastify/fast-json-stringify/pull/683 but this affects the stack at different layers, apparently. Basic example:

fastify.get('/y', {
  schema: {
    response: {
      200: Type.Object({
        x: Type.String({ format: 'date-time' }),
      })
    }
  }
}, () => {
  return {
    x: new Date()
  }
})

This produces this TypeScript error:

server/fastify.ts:77:3 - error TS2345: Argument of type '() => { x: Date; }' is not assignable to parameter of type 'RouteHandlerMethod<Server<typeof IncomingMessage, typeof ServerResponse>, IncomingMessage, ServerResponse<IncomingMessage>, ... 4 more ..., FastifyBaseLogger>'.
  Type '{ x: Date; }' is not assignable to type 'void | { x: string; } | Promise<void | { x: string; }>'.
    Type '{ x: Date; }' is not assignable to type '{ x: string; }'.
      Types of property 'x' are incompatible.
        Type 'Date' is not assignable to type 'string'.

77   () => {
     ~~~~~~~

But the code works as you'd expect - fast-json-stringify is able to stringify that Date object into an ISO Date string. fast-json-stringify has an explicit list of these conversions and they work, but they aren't supported at this level.

I think the core of this issue is that this module is using Static from TypeBox to derive types. But that is strict: the static type of a Type.String is just a string:

https://github.com/sinclairzx81/typebox/blob/fd1056b367479c7a9925143641d272b4a238ffad/src/type/string/string.ts#L77-L81

There's a StaticEncode type in TypeBox, but that's for custom types, not for defaults.

Anyway, this is a rough issue if you're using a database setup that produces JavaScript Date objects for date columns and you've been relying on their default stringification. And it works, but the types don't work and there's no easy workaround.

I'm open to suggestions from the maintainers about how this could be resolved. Off the top of my head I'm guessing:

Pull the type coercion tricks from fast-json-stringify into a type that this can import, or can live in this module.
Try to submit a PR upstream to Typebox (but this doesn't seem like it'd be accepted because the output coercion isn't a Typebox thing)
Some secret third thing?

Steps to Reproduce

Try using default stringification with a body, like:

fastify.get('/y', {
  schema: {
    response: {
      200: Type.Object({
        x: Type.Number({ format: 'date-time' }),
      })
    }
  }
}, () => {
  return {
    x: new Date()
  }
})

Expected Behavior

The types for this module should reflect what's possible with the code.

khokm commented 5 months ago

Struggle with that problem too, using this solution for now.

mcollina commented 5 months ago

@sinclairzx81 wdyt? How can we solve this?

sinclairzx81 commented 5 months ago

@mcollina Hi!

Automatic type coercion can be achieved with TypeBox Transforms + overriding the Fastify preSerialization and validationCompiler request/response phases. This hasn't been integrated into the provider yet (but there has been a few PR's attempted, see https://github.com/fastify/fastify-type-provider-typebox/pull/99 and https://github.com/fastify/fastify-type-provider-typebox/pull/127).

If it's helpful, I've updated the provider submitted on https://github.com/fastify/fastify-type-provider-typebox/pull/99 to work with the latest versions of Fastify. I'll post this below.

Experimental TypeProvider

The provider is implemented as a single file that can be copied into a project as a standalone module. This code may constitute a future TypeProvider (requiring a major semver)

Expand for provider.ts source code

```typescript import { FastifyInstance, FastifySchemaCompiler, FastifyTypeProvider, preValidationHookHandler } from 'fastify' import { StaticDecode, TSchema } from '@sinclair/typebox' import { TypeCompiler, TypeCheck } from '@sinclair/typebox/compiler' import { Value } from '@sinclair/typebox/value' export * from '@sinclair/typebox' /** Functions used to transform values during validation and preSerialization phases */ namespace TransformFunctions { const checks = new Map>() function ResolveCheck (schema: TSchema): TypeCheck { if (checks.has(schema)) return checks.get(schema)! checks.set(schema, TypeCompiler.Compile(schema)) return checks.get(schema)! } /* Converts request params, querystrings and headers into their target type */ export function ProcessValue (httpPart: string | undefined, schema: T, value: unknown) { const converted = httpPart === 'body' ? value : Value.Convert(schema as TSchema, value) const defaulted = Value.Default(schema, converted) const cleaned = Value.Clean(schema, defaulted) return cleaned } /* Generates errors for the given value. This function is only called on decode error. */ export function Errors (schema: T, value: unknown) { return [...ResolveCheck(schema).Errors(value)].map((error) => { return { message: `${error.message}`, instancePath: error.path } }) } /** Decodes a value or returns undefined if error */ export function Decode (schema: T, value: unknown): unknown { try { return ResolveCheck(schema).Decode(value) } catch { return undefined } } /** Encodes a value or throws if error */ export function Encode (schema: T, value: unknown) { return ResolveCheck(schema).Encode(value) } } const TypeBoxValidatorCompiler: FastifySchemaCompiler = ({ schema, httpPart }) => { return (value): any => { const processed = TransformFunctions.ProcessValue(httpPart, schema, value) const decoded = TransformFunctions.Decode(schema, processed) return decoded ? { value: decoded } : { error: TransformFunctions.Errors(schema, processed) } } } const TypeBoxPreSerializationHook: preValidationHookHandler = (...args: any[]) => { const [request, reply, payload, done] = args const response = request.routeOptions.schema.response const schema = response ? response[reply.statusCode] : undefined try { return schema !== undefined ? done(null, TransformFunctions.Encode(schema, payload)) : done(null, payload) // no schema to encode } catch (error) { done(error, null) } } /** `[Experimental]` Specialized Type Provider that supports Transform type decoding */ export interface TypeBoxTransformDecodeProvider extends FastifyTypeProvider { output: this['input'] extends TSchema ? StaticDecode : unknown } /** `[Experimental]` Configures a Fastify instance to use the TypeBox Transform infrastructure. */ export function TypeProvider (instance: T) { return instance.withTypeProvider() .addHook('preSerialization', TypeBoxPreSerializationHook) .setValidatorCompiler(TypeBoxValidatorCompiler) } ```

Experimental TypeProvider Usage

This is an example implementation using the provider above. It automatically converts numeric timestamps into JS Date objects by way of Transform types. Note that the configuration of the preSerialization and validationCompiler is handled by way of the TypeProvider function. Also note that this function also sets up the useTypeProvider inference.

import Fastify from 'fastify'
import { Type, TypeProvider } from './provider' // from above

// Function to configure validationCompiler and preSerialization & TypeProvider
const fastify = TypeProvider(Fastify())

// Transforms Number timestamp into a Date object.
const Timestamp = Type.Transform(Type.Number())
  .Decode(value => new Date(value))     // runs on -> validationCompiler phase
  .Encode(value => value.getTime())     // runs on -> preSerialization phase

// Route: Note that the timestamp is observed as Date object within the route.
fastify.post('/', {
  schema: {
    body: Timestamp,
    response: {
      200: Timestamp
    }
  }
}, (req, reply) => {
  console.log('server body:', req.body)    // ... receive Date object
  reply.send(new Date())                   // ... send Date object
})

// Client: Note that client must send Number for the request.
fastify
  .inject()
  .post('/').headers({ 'Content-Type': 'application/json' })
  .body(new Date().getTime())            // ... client sends number timestamps
  .then(response => {
    console.log(response.payload)        // ... client receives number timestamps
  })

I may be able to take another look at pushing this functionality through later in the year, but it may be a good opportunity for someone in the community to have a go also (I'd certainly be happy to review a PR if someone wants to take the above provider and integrate it). In the short term though, I would recommend a bit of experimentation with the provider implementation submitted here (just to make sure it ticks all the boxes, and to check that the overriding of request/response phases doesn't have unintended breakage in plugins)

Also, need to compare the implementation submitted here with future proposals to update the provider inference, just referencing this PR https://github.com/fastify/fastify/pull/5315 which would have implication for this and other providers.

Let me know if the above helps. Happy to field any additional questions on the implementation :) Cheers! S

sinclairzx81 commented 5 months ago

cc @ehaynes99

Bram-dc commented 5 months ago

Why is preSerialization hook used instead of setSerializerCompiler?

sinclairzx81 commented 5 months ago

@Bram-dc Hi,

Why is preSerialization hook used instead of setSerializerCompiler?

I believe the intent here was to apply the transform / encode "before" the value was sent for serialization. The preSerialization hook seemed the best way to achieve this.

fastify / fastify-type-provider-typebox