sinclairzx81 / typebox

Json Schema Type Builder with Static Type Resolution for TypeScript
Other
5.09k stars 162 forks source link

Introduce Type.StrictObject for automatic additionalProperties: false handling #1059

Closed arthurfiorette closed 2 weeks ago

arthurfiorette commented 3 weeks ago

Problem

A primary use case for this library is generating JSON schemas to serve as a single source of truth for validation libraries, such as AJV. These validation libraries are widely used for validating HTTP input from web applications.

When a schema is strictly typed, it visually suggests that only specific properties are allowed:

const T = Type.Object({
  x: Type.Number(),
  y: Type.Number(),
  z: Type.Number()
})

This implies that only x, y, and z should be present at runtime, a restriction that current typings support.

However, without explicitly setting additionalProperties: false for each Type.Object instance, schemas will allow extra properties, meaning { x: 1, y: 1, z: 1, a: 1 } would be considered valid, and validators would not strip out the extra a property.

This issue often goes unnoticed by users (many companies make this oversight), as typings and validations still work for core cases (e.g., { x: 'not a number' } fails validation). Common usage patterns, like the following, are especially vulnerable:

prisma.model.create({ data: request.body })
// or similar patterns

This scenario leads to Overposting or Mass Assignment vulnerabilities, which most users are unaware of.

While AJV offers a removeAdditional: 'all' option, it does not work as expected, as is also the case with similar validators. (AJV is the primary example here because it is what Fastify currently uses.)

Manually adding { additionalProperties: false } across thousands of schema declarations is tedious, error-prone, and detrimental to the developer experience (DX).

Proposed Solution

Similar to how Zod offers z.object().strict() or z.strictObject(), TypeBox could introduce Type.StrictObject() (or a shorter version, Type.SObject()) to function like Type.Object but with { additionalProperties: false } automatically applied.

The implementation should also handle intersecting types gracefully, as outlined here.

Alternative approaches to achieve the same outcome may also be worth exploring.

sinclairzx81 commented 2 weeks ago

@arthurfiorette Hi, Sorry for the late reply on this.


AdditionalProperties

Similar to how Zod offers z.object().strict() or z.strictObject(), TypeBox could introduce Type.StrictObject() (or a shorter version, Type.SObject()) to function like Type.Object but with { additionalProperties: false } automatically applied.

Unfortunately, TypeBox isn't able to apply additionalProperties: false as standard (or via standard StrictObject object) as the keyword is currently being considered to express the following TS construct.

// TypeScript
type T = { [key: string]: number } // all string keys, 

// TypeBox
const T = Type.Object({
  [Type.StringKey]: Type.Number()
})

// Json Schema
const T = {
  type: "object",
  properties: {}
  additionalProperties: { type: 'number' } // additionalProperties also support schemas
}

The additionalProperties keyword is being considered (along with patternProperties) to provide more aligned representation for Record types (as per the above TS syntax). Keep in mind that the additionalProperties keyword isn't limited to just false, rather it accepts any valid Json Schema being applied there.

The work to have TypeBox reconcile for both false and TSchema for compositional types like Intersect has proved untenably difficult in the past (it has been attempted before), this is mostly due to some of the semantics in Json Schema around additionalProperties, and that Intersect types support more than just TObject.

Have a read over the following link it get a bit more insight into Json Schema semantics.

https://json-schema.org/understanding-json-schema/reference/object#extending


StrictObject

While TypeBox can't provide a StrictObject type as part of the library. Users can implement their own version of it. The following achieves this.

TypeScript Link Here

import { Type, Static, TProperties, TObject } from '@sinclair/typebox'

const StrictObject = <Properties extends TProperties>(properties: Properties): TObject<Properties> => {
  return Type.Object(properties, { additionalProperties: false })
}

const T = StrictObject({
  x: Type.Number(),
  y: Type.Number(),
  z: Type.Number()
})

type T = Static<typeof T>

StrictIntersect

As per the "Extending Closed Schema" link above, applying additionalProperties false to Objects introduces issues with respect to Intersect (allOf) and some ambiguities with respect to how to reconcile for these cases. As a possible interpretation to this problem, the following implements a StrictIntersect type that will apply unevaluatedProperties: false if any of the types passed include a additionalProperties: false.

const StrictIntersect = <Types extends TObject[]>(types: [...Types]): TIntersect<Types> => {
  // remove additionalProperties from all types
  const mapped = types.map(type => {
    const { additionalProperties, ...rest } = type // remove additionalProperties
    return rest
  }) as Types
  // if any type has a closed schema ...
  const isClosed = types.some(type => type.additionalProperties === false)
  // ... then construct unevaluatedProperties accordingly
  const unevaluatedProperties = isClosed
    ? { unevaluatedProperties: false }
    : {}
  return Type.Intersect(mapped, { ...unevaluatedProperties }) as never
}

const A = StrictObject({
  a: Type.Number(),
  b: Type.Number()
})

const B = StrictObject({
  c: Type.Number(),
  d: Type.Number()
})

// S will have unevaluatedProperties if any
// sub type has additionalProperties false.
const S = StrictIntersect([A, B])

Note that there is room for interpretation here. The following will apply unevaluatedProperties if ANY sub schema includes additionalProperties: false. This logic could be updated to such that the unevaluatedProperties constraint is only applied when ALL of the sub schemas include this constraint. Json Schema doesn't have anything to say about which is the correct interpretation, so because of this, TypeBox doesn't either.

This said, you should be able to use StrictObject and StrictIntersect in your application, but just with the knowledge that you're working a little bit above and beyond the specification.


Will close out this issue for now. If you have any questions on the above, feel free to ping on this link

Cheers! S

arthurfiorette commented 2 weeks ago

Hi @sinclairzx81, thanks a lot for your detailed explanation!

Does not using Type.StringKey and Type.StrictObject() mean that we want to achieve different things?

I'd argue when using Type.StringKey inside a Type.StrictObject() to simply not do anything or rather throw an error.

Our use case is against multiple packages and even across multiple repos and even owners (sub packages and dependencies from other authors as well) and enforcing all of them to manually write their own StrictObject does not seem suitable.

Since Type.StringKey is not meant at all to be used alongside StrictObject and using a different Kind for StrictObject could make Type.Intersect understand it (or just also add a Type.StrictIntersect), I still believe the best solution for this is to have something built in, would it be possible?

Also, as you said it was attempted before and a fix-all solution wasn't found, as long its limitations are properly documented, there's no need to solve all use cases as well for now.

arthurfiorette commented 1 week ago

cc @sinclairzx81