Is it possible to write a mutually recursive schema?

[Gnuxie]

I tried to use Type.Index on a recursive type ie using Type.Index on This as a hack. But this didn't seem to work with StaticDecode and all the indexed types were inferred as never

[sinclairzx81]

Hi @Gnuxie,

Unfortunately, Type.Index isn't well supported with Recursive This as of today. The reason is due to This being just a { $ref: '...' } self referential schema, and there isn't a way to dereference the $ref inside a recursive callback (because the target of $ref isn't constructed until the callback returns).

There are plans underway to try and provide better options around this in the next significant release, however the work to achieve recursive deferred indexed access types is quite vast and requires re-architecting some of TypeBox's type inference internals. Updates here won't be ready for some time, but it is being worked on.

---

Is it possible to write a mutually recursive schema?

Yes, it is possible, however this functionality is not built into the library currently, however it is a feature planned for the next main release. In the interim, I have actually published a technical preview of this functionality as a prototype. You can find it at the link below.

https://github.com/sinclairzx81/typebox/blob/master/example/prototypes/module.ts

You will need to copy and paste this module into your project, the following is the usage.

import { Type, StaticDecode } from '@sinclair/typebox'
import { Module, ModuleRef } from './prototypes/module'

// Mutual Recursive Types need to be embedded within a Module that allows cyclic references 
// to be dereferenced irrespective of topological ordering. We use a special case ModuleRef
// for interior referencing (a facade to Ref)
const M = Module({
  A: Type.Object({
    type: Type.Literal('A'),
    b: ModuleRef('B')
  }),
  B: Type.Object({
    type: Type.Literal('B'),
    c: ModuleRef('C')
  }),
  C: Type.Object({
    type: Type.Literal('C'),
    a: ModuleRef('A')
  })
})

const A = M.Import('A') // { $defs: { A: ..., B: ..., C: ... }, $ref: 'A' }
const B = M.Import('B')
const C = M.Import('C')

type A = StaticDecode<typeof A>
type B = StaticDecode<typeof B>
type C = StaticDecode<typeof A>

function test(value: A) {
  value.b.c.a.b.c.a.b.c.a.b.c
}

The Type.Module is currently published as a prototype to get community feedback on. It isn't the final design for the feature, but should be ok to use (and I anticipate updates to the final API being relatively trivial). I am very much open to community feedback on this design, so if you have any thoughts of suggestions for improvement, let me know.

Hope this helps! S

[Gnuxie]

That's so cool, I'll try this out as soon as I get back to my hobby project and I'll let you know what I think to it. Awesome work once again!

[sinclairzx81]

@Gnuxie All good :)

Will close off this issue for now, but let me know how you get on with the module / mutual recursion prototype. As mentioned, very keen to get feedback on it, so let me know your thoughts.

All the best! S

[Gnuxie]

oki, so I had a go with it.

First thing to note is that I noticed that there's nothing on the type level to warn you if you create a typo while writing ModuleRef, I haven't figured out what will happen if i did that at runtime yet though.

There might be a way around that if ModuleRef got provided via an argument in a callback, and from there you can probably make it so that the parameter of ModuleRef is typed to only allow the properties defined in the module. I'm not sure though and this is already does the job, and that might be considered noisy for little benefit. Thanks again, I'll come back as I keep playing with it.

Module((ModuleRef) => ({
  MethodDescription: Type.Object(
    {
      slots: Type.Array(ModuleRef('SlotDescription')), // would error at the type level because the only property at the minute is `MethodDescription`.
      expression: ModuleRef("MethodExpression"),
    }
  ),
}));

[sinclairzx81]

@Gnuxie Hi, thanks for the feedback :)

There might be a way around that if ModuleRef got provided via an argument in a callback, and from there you can probably make it so that the parameter of ModuleRef is typed to only allow the properties defined in the module. I'm not sure though and this is already does the job, and that might be considered noisy for little benefit. Thanks again, I'll come back as I keep playing with it.

Yeah, have explored this aspect quite a bit. The problem I ran into was that TS can't resolve properties keys within the context of a object literal. It can partially resolve within the context of a class however (as classes have access to the this context), but it still doesn't work quite right when used for establishing left side return types as the class signatures become cyclic when interior class members reference themselves....the issue is somewhat analogous to the following.

// error: Return type annotation circularly references itself.
declare function X(): ReturnType<typeof X> 

The trick to making this work was to treat the ModuleRef string parameter as a kind of latent query to deference the type "after the fact". By making the string unaware of it's surroundings, you can mitigate cyclic referencing issues, and enable mutual recursion, but it does come at a cost of not being able to provide type hints of possible things to reference. I think it's ok tho, because if the ModuleRef string is incorrect, TypeBox can infer the type as TNever or just throw an error on calls to .Import().

TypeBox is working towards this design in the next revision, and I have been doing quite a bit of performance tuning around it. A combination of deferred inference (as per ModuleRef (which will be Ref in the next release) and tail call optimized conditional types seems to yield really good scalable types. Will be looking for people to help me test that on existing codebases before the release, so if you're interested, I could use some help when things are a little closer to release.

At this stage, I had been hoping to get the next release out before the end of the year, but there are some other significant updates in the release (which I have been taking my time to get right), as well as exploring some very advanced new features I think people will like. With a bit of luck I'll get these updates out before the end of the year, and with that will come a push for TypeBox 1.0 in 2025 (fingers crossed)

Thanks again for the feedback! S

[sinclairzx81]

@Gnuxie Hi,

Just a heads up, Module Types have just been published on 0.34.0. Documentation on the feature can be found at the following URL.

https://github.com/sinclairzx81/typebox#types-modules

This feature was pushed ahead inline with the new Syntax Types feature which supports TS module parsing.

https://github.com/sinclairzx81/typebox#syntax-types

And to provide future backend for the following project (with a plan to replace the TS compiler for code transformation)

https://sinclairzx81.github.io/typebox-workbench

The new features have taken quite a bit of research to achieve, so always open to feedback from the community on them. If you get a chance to use them and find any bugs, feel free to ping an issue (I expect these features will be a focus for the entirety of the 0.34.0 revision)

Cheers! S

[Gnuxie]

Excellent, that's great news, thank you for the update. I probably won't get to play around again for some time but I'll get back to you if I get the opportunity!