sinclairzx81 / typebox

Json Schema Type Builder with Static Type Resolution for TypeScript
Other
4.79k stars 153 forks source link

Issue when using Type.Unsafe in an object in conjunction with Type.Composite #639

Closed danecando closed 11 months ago

danecando commented 11 months ago

When using Type.Unsafe<Date>({ [Kind]: "Date" }) in an object to represent a date

https://github.com/fastify/fastify/discussions/3357#discussioncomment-4241449

It works fine until I try to combine another object with Type.Composite

For context the createSchema functions is generating typebox schemas from a drizzle db schema, the object that is the second arg allows for overriding fields


Types

export const userSchema = createSelectSchema(users, {
  // lastLoggedIn: Type.Unsafe<Date>({ [Kind]: "Date" }),
  // createdAt: Type.Unsafe<Date>({ [Kind]: "Date" }),
  // updatedAt: Type.Unsafe<Date>({ [Kind]: "Date" }),
});

console.log("userSchema", userSchema);

export const userRecordSchema = Type.Composite([userSchema, relationsSchema]);

console.log("userRecordSchema", userRecordSchema);

Logs

userSchema {
api:dev:   type: 'object',
api:dev:   properties: {
api:dev:     id: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     firebaseId: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     email: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     displayName: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     discriminator: { type: 'number', [Symbol(TypeBox.Kind)]: 'Number' },
api:dev:     lastLoggedIn: { type: 'Date', [Symbol(TypeBox.Kind)]: 'Date' },
api:dev:     bio: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     avatarUrl: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     anniversary: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     location: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     active: { type: 'boolean', [Symbol(TypeBox.Kind)]: 'Boolean' },
api:dev:     banned: { type: 'boolean', [Symbol(TypeBox.Kind)]: 'Boolean' },
api:dev:     homegroupId: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     programId: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     createdAt: { type: 'Date', [Symbol(TypeBox.Kind)]: 'Date' },
api:dev:     updatedAt: { type: 'Date', [Symbol(TypeBox.Kind)]: 'Date' }
api:dev:   },
api:dev:   required: [
api:dev:     'id',            'firebaseId',
api:dev:     'email',         'displayName',
api:dev:     'discriminator', 'lastLoggedIn',
api:dev:     'bio',           'avatarUrl',
api:dev:     'anniversary',   'location',
api:dev:     'active',        'banned',
api:dev:     'homegroupId',   'programId',
api:dev:     'createdAt',     'updatedAt'
api:dev:   ],
api:dev:   [Symbol(TypeBox.Kind)]: 'Object'
api:dev: }
api:dev: userRecordSchema {
api:dev:   type: 'object',
api:dev:   properties: {
api:dev:     id: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     firebaseId: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     email: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     displayName: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     discriminator: { type: 'number', [Symbol(TypeBox.Kind)]: 'Number' },
api:dev:     lastLoggedIn: { type: 'Date', [Symbol(TypeBox.Kind)]: 'Date' },
api:dev:     bio: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     avatarUrl: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     anniversary: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     location: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     active: { type: 'boolean', [Symbol(TypeBox.Kind)]: 'Boolean' },
api:dev:     banned: { type: 'boolean', [Symbol(TypeBox.Kind)]: 'Boolean' },
api:dev:     homegroupId: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     programId: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     createdAt: { type: 'Date', [Symbol(TypeBox.Kind)]: 'Date' },
api:dev:     updatedAt: { type: 'Date', [Symbol(TypeBox.Kind)]: 'Date' },
api:dev:     homegroup: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     program: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' }
api:dev:   },
api:dev:   required: [
api:dev:     'id',            'firebaseId',
api:dev:     'email',         'displayName',
api:dev:     'discriminator', 'lastLoggedIn',
api:dev:     'bio',           'avatarUrl',
api:dev:     'anniversary',   'location',
api:dev:     'active',        'banned',
api:dev:     'homegroupId',   'programId',
api:dev:     'createdAt',     'updatedAt',
api:dev:     'homegroup',     'program'
api:dev:   ],
api:dev:   [Symbol(TypeBox.Kind)]: 'Object'
api:dev: }

Types

export const userSchema = createSelectSchema(users, {
  lastLoggedIn: Type.Unsafe<Date>({ [Kind]: "Date" }),
  createdAt: Type.Unsafe<Date>({ [Kind]: "Date" }),
  updatedAt: Type.Unsafe<Date>({ [Kind]: "Date" }),
});

console.log("userSchema", userSchema);

export const userRecordSchema = Type.Composite([userSchema, relationsSchema]);

console.log("userRecordSchema", userRecordSchema);

const userJsonSchema = createSelectSchema(users, {
  lastLoggedIn: Type.String(),
  createdAt: Type.String(),
  updatedAt: Type.String(),
});

Logs

userSchema {
api:dev:   type: 'object',
api:dev:   properties: {
api:dev:     id: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     firebaseId: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     email: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     displayName: { type: 'string', [Symbol(TypeBox.Kind)]: 'String' },
api:dev:     discriminator: { type: 'number', [Symbol(TypeBox.Kind)]: 'Number' },
api:dev:     lastLoggedIn: { [Symbol(TypeBox.Kind)]: 'Date' },
api:dev:     bio: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     avatarUrl: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     anniversary: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     location: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     active: { type: 'boolean', [Symbol(TypeBox.Kind)]: 'Boolean' },
api:dev:     banned: { type: 'boolean', [Symbol(TypeBox.Kind)]: 'Boolean' },
api:dev:     homegroupId: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     programId: { anyOf: [Array], [Symbol(TypeBox.Kind)]: 'Union' },
api:dev:     createdAt: { [Symbol(TypeBox.Kind)]: 'Date' },
api:dev:     updatedAt: { [Symbol(TypeBox.Kind)]: 'Date' }
api:dev:   },
api:dev:   required: [
api:dev:     'id',            'firebaseId',
api:dev:     'email',         'displayName',
api:dev:     'discriminator', 'lastLoggedIn',
api:dev:     'bio',           'avatarUrl',
api:dev:     'anniversary',   'location',
api:dev:     'active',        'banned',
api:dev:     'homegroupId',   'programId',
api:dev:     'createdAt',     'updatedAt'
api:dev:   ],
api:dev:   [Symbol(TypeBox.Kind)]: 'Object'
api:dev: }
api:dev: userRecordSchema { type: 'object', properties: {}, [Symbol(TypeBox.Kind)]: 'Object' }
sinclairzx81 commented 11 months ago

@danecando Hi,

This is a somewhat known issue when composing Unsafe with some of TypeBox's more complex types (specifically Composite and Index). The problem relates to a need to check that the internal schematics being composited are valid, and Unsafe types are considered to be invalid unless they meet the following criteria.

The following are 3 options you can try


Option 1: Rename Kind and Register on TypeRegistry

The following renames the Kind from Date to UnsafeDate to avoid conflicts with Type.Date(). This type is then registered on the TypeRegistry which allows the Unsafe type to pass Composite type validation checks. You can optionally implement the callback to also have TypeBox check values of UnsafeDate.

See here for documentation on the TypeRegistry.

import { Type, TypeRegistry } from '@sinclair/typebox'

TypeRegistry.Set('UnsafeDate', () => true) // required

export const userSchema = createSelectSchema(users, {
  lastLoggedIn: Type.Unsafe<Date>({ [Kind]: "UnsafeDate" }),
  createdAt: Type.Unsafe<Date>({ [Kind]: "UnsafeDate" }),
  updatedAt: Type.Unsafe<Date>({ [Kind]: "UnsafeDate" }),
});

export const userRecordSchema = Type.Composite([userSchema, relationsSchema]); // should be ok

Option 2: Use Intersect instead of Composite

You can use Type.Intersect instead of Type.Composite as both return the same semantic type. Intersect is able to avoid the is-valid check as it does not need to evaluate interior types to produce a the singular object type. The downside however is that the schematics will be represented with the Json Schema allOf keyword.

import { Type } from '@sinclair/typebox'

export const userSchema = createSelectSchema(users, {
  lastLoggedIn: Type.Unsafe<Date>({ [Kind]: "UnsafeDate" }),
  createdAt: Type.Unsafe<Date>({ [Kind]: "UnsafeDate" }),
  updatedAt: Type.Unsafe<Date>({ [Kind]: "UnsafeDate" }),
});

export const userRecordSchema = Type.Intersect([userSchema, relationsSchema]); // should be ok

Option 3: Cast Known Type in Unsafe

You can use Unsafe to cast a known type. This might be the most appropriate (and simplest) option if you're using Fastify. With this approach, you don't need to register the Unsafe type on the TypeRegistry as the interior type String is a known built in type.

import { Type } from '@sinclair/typebox'

export const userSchema = createSelectSchema(users, {
  lastLoggedIn: Type.Unsafe<Date>(Type.String({ format: 'date-time' })),
  createdAt: Type.Unsafe<Date>(Type.String({ format: 'date-time' })),
  updatedAt: Type.Unsafe<Date>(Type.String({ format: 'date-time' })),
});

export const userRecordSchema = Type.Composite([userSchema, relationsSchema]); // should be ok

Considerations

As per https://github.com/fastify/fastify/discussions/3357#discussioncomment-4241449, there is some complexity with handling string encoded dates and having them reflect back as actual JavaScript Date objects. I don't believe Fastify will automatically convert iso date strings into Date objects for you, so the value you will see in the route for the above types will be Date, but the actual value will be of type string. Just be a bit mindful of this aspect.

Hope this helps S

danecando commented 11 months ago

@sinclairzx81 thanks for the response and also thanks for your work. TypeBox is really, really awesome :)

It looks like #3 is going to work for me Type.Unsafe<Date>(Type.String()). When I get cute with the types like adding format: 'date-time' fastify yells at me. I think it's the stringifier and not Ajv. Seems weird to complain about it if it knows how to stringify Date objects but anyways not your problem.

My goal is to build an app that consists of a fastify API server, and API client library, and a NextJS web app. Since the types don't get encoded in the json response (something like superjson) and the entire codebase will have access to the typebox schemas, the plan was to use it with Encode and Decode on the client to decode the json.

If you think that's a dumb idea let me know I have those sometimes. Seems like taking advantage of the schemas in this way could provide reasonable e2e type safety though

PS: feel free to close this of course

sinclairzx81 commented 11 months ago

@danecando Hi,

My goal is to build an app that consists of a fastify API server, and API client library, and a NextJS web app. Since the types don't get encoded in the json response (something like superjson) and the entire codebase will have access to the typebox schemas, the plan was to use it with Encode and Decode on the client to decode the json.

The TypeBox Transform Types (Encode and Decode) were actually written Fastify in mind. If you're interested, I've actually written a prototype implementation which integrates Transform types into Fastify's Request/Response pipeline (which automatically handles Encode and Decode + Type Inference). You can find that module at the link below (which you are free to copy and paste into a project and test)

https://github.com/sinclairzx81/fastify-type-provider-typebox/blob/main/src/transform.ts

This can be used as follows.

TypeScript Link Here

import { TypeBoxTransformProvider } from './transform'
import { Type } from '@sinclair/typebox'
import Fastify from 'fastify'

const fastify = TypeBoxTransformProvider(Fastify())

// Timestamp Transform
const Timestamp = Type.Transform(Type.Number())
  .Decode(value => new Date(value))      // number > Date
  .Encode(value => value.getTime())      // Date > number

// Route
fastify.post('/date', {
  schema: {
    body: Timestamp,                      // number
    response: { 200: Timestamp }          // number
  }
}, (req, res) => {
  const { body } = req.body               // Date
  res.send(body)                          // Date
})

It would be possible to implement a client version of this if the client shared the same schematics as the server. Implementing both client and server ends would make for a great open source project.

Will close off this issue for now. All the best! S