References Array Documentation

[gotjoshua]

I notice that many functions accept an optional array of "references" i see this mentioned in the changelog: https://github.com/sinclairzx81/typebox/blob/078cfacd44336aa01528c3241658516d7f2ec46e/changelog.md?plain=1#L181

I don't find any examples of how to use this optional param.

[sinclairzx81]

@gotjoshua Hi,

The Partial, Required Omit, Pick and KeyOf cannot currently contain a Type.Ref(). This functionality was temporarily disabled (albeit several months ago) to make provisions for a new compositing model (which is still a work in process). There are plans to re-enable auto deref for utility types, but only once the new compositor system in place (which is being planned for the next minor release)

The following are some examples of what is and isn't possible in the current release. Note optional is supposed for referenced types, but partial is not.

Possible

const Vector = Type.Object({
  x: Type.Number(),
  y: Type.Number(),
  z: Type.Number(),
}, { $id: 'Vector' })

const Vertex = Type.Object({
  // ok: optional is exterior and doesn't require internal deref.
  position: Type.Optional(Type.Ref(Vector)) 
})

Not Possible

const Vector = Type.Object({
  x: Type.Number(),
  y: Type.Number(),
  z: Type.Number(),
}, { $id: 'Vector' })

const PartialVector = Type.Partial(Type.Ref(Vector)) // error: operation requires internal deref

Hope this helps S

[gotjoshua]

Thanks @sinclairzx81

I think this is a different question.

I am referring to the function signatures of many functions eg compile (which in turn passes the references to assert)

https://github.com/sinclairzx81/typebox/blob/078cfacd44336aa01528c3241658516d7f2ec46e/src/compiler/compiler.ts#L473-L474

I can't find examples of when it is meaningful to use this function arg.

Thanks for your generous engagement and willingness to help!

Maybe i can offer a PR for the docs, once i manage to understand your replies...

[sinclairzx81]

@gotjoshua Oh, sorry, I misread your question.

The references array is used to specify auxiliary referenced schemas. When using Type.Ref(...), the resulting schema is represented as { $ref: 'T' } (meaning a reference to some schema with an $id of T). Because the compiler needs to resolve the referenced schema somehow, this is made possible by passing the referenced schema on the references array (which is used as a lookup internal to the compiler).

Referenced Schemas

The following shows the schema outputs for the following Vector and Vertex types. Note that the Vertex type is referencing Vector for position and normal. For the Vertex type to compile, you must specify the Vector on the references array.

import { TypeCompiler } from '@sinclair/typebox/compiler'
import { Type } from '@sinclair/typebox'

const Vector = Type.Object({       // const Vector = {
  x: Type.Number(),                //   $id: 'Vector',
  y: Type.Number(),                //   type: 'object',
  z: Type.Number(),                //   required: ['x', 'y', 'z'],
}, { $id: 'Vector' })              //   properties: {
                                   //     x: { type: 'number' },
                                   //     x: { type: 'number' },
                                   //     x: { type: 'number' }
                                   //   }
                                   // }

const Vertex = Type.Object({       // const Vertex = {
   position: Type.Ref(Vector),     //   $id: 'Vertex',
   normal: Type.Ref(Vector)        //   type: 'object',
}, { $id: 'Vertex' })              //   required: ['position', 'normal'],
                                   //   properties: {
                                   //     position: { $ref: 'Vector' },                           
                                   //     normal: { $ref: 'Vector' },
                                   //   }
                                   // }

const C = TypeCompiler.Compile(Vertex, [Vector]) // passing the Vector schema is required!!!

Inlined Schemas

Just for comparison, without using Type.Ref(...), the default for TypeBox is to just inline schemas. The following shows the expanded schema for Vertex which contains all the information required to compile the type. (so references is not required in this scenario)

import { TypeCompiler } from '@sinclair/typebox/compiler'
import { Type } from '@sinclair/typebox'

const Vector = Type.Object({       // const Vector = {
  x: Type.Number(),                //   type: 'object',
  y: Type.Number(),                //   required: ['x', 'y', 'z'],
  z: Type.Number()                 //   properties: {
})                                 //     x: { type: 'number' },
                                   //     y: { type: 'number' },
                                   //     z: { type: 'number' }
                                   //   }
                                   // }

const Vertex = Type.Object({       // const Vertex = {
   position: Vector,               //   type: 'object',
   normal: Vector                  //   required: ['position', 'normal'],
})                                 //   properties: {
                                   //     position: {
                                   //       type: 'object',
                                   //       required: ['x', 'y', 'z'],
                                   //       properties: {
                                   //         x: { type: 'number' },
                                   //         y: { type: 'number' },
                                   //         z: { type: 'number' }
                                   //       }
                                   //     },                            
                                   //     normal: {
                                   //       type: 'object',
                                   //       required: ['x', 'y', 'z'],
                                   //       properties: {
                                   //         x: { type: 'number' },
                                   //         y: { type: 'number' },
                                   //         z: { type: 'number' }
                                   //       }
                                   //     }, 
                                   //   }
                                   // }

const C = TypeCompiler.Compile(Vertex) // no referencing, so ok

Inline vs Reference

By default, TypeBox prefers to use inline schemas, however referenced schemas can be useful for schema publishing scenarios where you want to avoid duplication of schematics that represent the same type (like Vector in the above example).

Future

There is some consideration to deprecating the references array by internally caching schemas in subsequent revisions of TypeBox. Historically Ive been somewhat against internally caching schemas for the user, but there are memory saving optimizations in the compiler that can be achieved via auto deref / caching, so I have been reviewing this aspect again. For now, I'd probably hold off on explicit documentation for the current revision, but just be mindful that auto deref may be a feature in later versions.

Hope this helps! S

[gotjoshua]

Hope this helps!

amazing, very thorough response, thanks !!