sinclairzx81 / typebox

Json Schema Type Builder with Static Type Resolution for TypeScript
Other
5.08k stars 161 forks source link

Visit meta type #1077

Closed zoriya closed 2 days ago

zoriya commented 1 week ago

I'd like to create a schema that looks like another complex one, but only has a few differences (replaces one type by another). I think the simplest idea for me would be to use a visitor and simply replace the schema.

Something like this pseudocode:

const SeedMovieTranslation = Type.Visit(MovieTranslation,
    (schema) => {
        if (schema.$id === Image.$id)
            return SeedImage;
        return schema;
    },
    { clone: true },
);

Since TypeGuards already exist, this feature could be pretty powerful quickly.

Type inference should be basically free because this function would just edit the schema in-place or clone it and edit the clone.

If you think this could be useful & could be part of typebox, I'll contribute it in the following days. lmk what you think of this

sinclairzx81 commented 1 week ago

@zoriya Hi! The Visit type seems like a cool idea!

Unfortunately, TypeBox can only support types that have direct TypeScript equivalents. The Type.* API is currently limited to:

While there are a few exceptions (like Transform), the API generally sticks closely to standard TypeScript types


This said, there is a place for community prototypes if you would like to submit one?

https://github.com/sinclairzx81/typebox/tree/master/example/prototypes

I would be quite keen to see a reference implementation of Visit (especially with the inference working). I usually use the TSP to draft up reference implementations (it's a good place to share and refine prototypes prior to submitting them)

Mostly TypeBox aims to make implementing types (like Visit) possible on top of the existing type system. However if a type isn't representable (but can be represented in TypeScript's type system), then that's usually a sign TypeBox needs updating to support the types creation, but not to implement the type outright (as TypeBox aims for generality). Implementing a Visit type on top of the current type system would be a good test of that.

Let me know if you would like to submit a Prototype Cheers! S

zoriya commented 2 days ago

oh yeah type inference doesn't work :c

idk why i thought it would work, i don't think this can work.

I'll leave the Visit function i did here in case wants to use it anyways.

export const Visit = (
    schema: TSchema,
    mapper: (schema: TSchema) => TSchema | undefined,
) => {
    schema = mapper(schema) ?? schema;

    if (KindGuard.IsUnion(schema)) {
        for (let i = 0; i < schema.anyOf.length; i++) {
            schema.anyOf[i] = Visit(schema.anyOf[i], mapper);
        }
    } else if (KindGuard.IsIntersect(schema)) {
        for (let i = 0; i < schema.anyOf.length; i++) {
            schema.allOf[i] = Visit(schema.allOf[i], mapper);
        }
    } else if (
        KindGuard.IsArray(schema) ||
        KindGuard.IsIterator(schema) ||
        KindGuard.IsAsyncIterator(schema)
    ) {
        // array needs to be before object
        schema.items = Visit(schema.items, mapper);
    } else if (KindGuard.IsTuple(schema)) {
        if (schema.items) {
            for (let i = 0; i < schema.items.length; i++) {
                schema.items[i] = Visit(schema.items[i], mapper);
            }
        }
    } else if (KindGuard.IsRecord(schema)) {
        // we can't visit record keys because keys are already transformed to strings.
        for (const [key, value] of Object.entries(schema.patternProperties)) {
            schema.patternProperties[key] = Visit(value, mapper);
        }
        if (schema.additionalProperties && schema.additionalProperties !== true) {
            schema.additionalProperties = Visit(schema.additionalProperties, mapper);
        }
    } else if (KindGuard.IsObject(schema)) {
        for (const key of Object.keys(schema.properties)) {
            schema.properties[key] = Visit(schema.properties[key], mapper);
        }
        if (schema.additionalProperties && schema.additionalProperties !== true) {
            schema.additionalProperties = Visit(schema.additionalProperties, mapper);
        }
    } else if (KindGuard.IsMappedKey(schema)) {
        // i have no idea how to traverse this ngl
    } else if (KindGuard.IsMappedResult(schema)) {
        for (const key of Object.keys(schema.properties)) {
            schema.properties[key] = Visit(schema.properties[key], mapper);
        }
    } else if (KindGuard.IsConstructor(schema) || KindGuard.IsFunction(schema)) {
        for (let i = 0; i < schema.parameters.length; i++) {
            schema.parameters[i] = Visit(schema.parameters[i], mapper);
        }
        schema.returns = Visit(schema.returns, mapper);
    } else if (KindGuard.IsNot(schema)) {
        schema.not = Visit(schema.not, mapper);
    } else if (KindGuard.IsPromise(schema)) {
        schema.item = Visit(schema.item, mapper);
    }
    return schema;
};
sinclairzx81 commented 2 days ago

@zoriya Hi!

Hey, thanks for the Visit code. I do think there is value in this functionality, especially as it relates to programmatic type remapping (the Visit function would share a lot in common with Mapped Types)

oh yeah type inference doesn't work :c idk why i thought it would work, i don't think this can work.

Yeah, its very difficult with the current capabilities of TypeScript. Here is some code that pushes things as far as they can probably go under the current capabilities of the language (as far as I know)

TypeScript Link Here

import { Type } from '@sinclair/typebox'
import * as Types from '@sinclair/typebox'

// TVisitProperties Requires a UnionToTuple to gather keys first, then Accumulated via TVisitPropertiesWithKey
type TVisitPropertiesWithKey<Properties extends Types.TProperties, PropertyKeys extends (keyof Properties)[], Result extends Types.TSchema[] = []> = (
  PropertyKeys extends [infer Left extends keyof Properties, ...infer Right extends (keyof Properties)[]]
   ? TVisitPropertiesWithKey<Properties, Right, [...Result, ...TVisit<Properties[Left]>]>
   : Result
)
type TVisitProperties<Properties extends Types.TProperties, PropertyKeys extends (keyof Properties)[] = Types.UnionToTuple<keyof Properties>> = (
  TVisitPropertiesWithKey<Properties, PropertyKeys>
)
// TVisitRest Accumulates across an Array (or Rest) of Types
type TVisitRest<Types extends Types.TSchema[], Result extends Types.TSchema[] = []> = (
  Types extends [infer Left extends Types.TSchema, ...infer Right extends Types.TSchema[]]
    ? TVisitRest<Right, [...Result, ...TVisit<Left>]>
    : Result 
)
// TVisit Accumulates Types, by pushing the Type first then Interior Type [Type, ...Interior]
type TVisit<Type extends Types.TSchema, Result extends Types.TSchema[] = [Type, ...(
  Type extends Types.TObject<infer Properties extends Types.TProperties> ? TVisitProperties<Properties> :
  Type extends Types.TConstructor<infer Parameters extends Types.TSchema[], infer InstanceType extends Types.TSchema> ? [...TVisitRest<Parameters>, ...TVisit<InstanceType>] :
  Type extends Types.TFunction<infer Parameters extends Types.TSchema[], infer ReturnType extends Types.TSchema> ? [...TVisitRest<Parameters>, ...TVisit<ReturnType>] :
  Type extends Types.TIntersect<infer Types extends Types.TSchema[]> ? TVisitRest<Types> :
  Type extends Types.TUnion<infer Types extends Types.TSchema[]> ? TVisitRest<Types> :
  Type extends Types.TTuple<infer Types extends Types.TSchema[]> ? TVisitRest<Types> :
  Type extends Types.TAsyncIterator<infer Type extends Types.TSchema> ? TVisit<Type> :
  Type extends Types.TIterator<infer Type extends Types.TSchema> ? TVisit<Type> :
  Type extends Types.TArray<infer Type extends Types.TSchema> ? TVisit<Type> :
  Type extends Types.TPromise<infer Type extends Types.TSchema> ? TVisit<Type> :
  []
)]> = Result 
// TVisitParameter takes a Union of all Types
type TVisitParameter<Types extends Types.TSchema[], Result extends Types.TSchema = never> = (
  Types extends [infer Left extends Types.TSchema, ...infer Right extends Types.TSchema[]]
    ? TVisitParameter<Right, Result | Left>
    : Result
)
// Callback for TVisit Mapper Function
type TVisitCallback<Type extends Types.TSchema> = (type: TVisitParameter<TVisit<Type>>) => unknown

function Visit<Type extends Types.TSchema, Callback extends TVisitCallback<Type>>(type: Type, callback: Callback): ReturnType<Callback> {
  // Todo: Visit Code Here, Dispatch on Callback
  throw 'Not implemented'
}

// Usage

const T = Type.Object({
  x: Type.Number(),
  y: Type.String(),
  z: Type.Object({
    tuple: Type.Tuple([
      Type.Literal(1), 
      Type.Literal(2), 
      Type.Literal(3)
    ])
  })
})

// The 'visited' parameter is a Union of all visit-able types.
const result = Visit(T, visited => {

  // Control flow analysis is possible for ReturnType - Potential for Mapping Interior Types 
  return Types.TypeGuard.IsLiteral(visited) ? visited : undefined
})

So, it is somewhat possible to apply control flow analysis to filter for a subset of types (the above filters for TLiteral values from the given type), however the visited parameter / argument needs to be typed as a TS union which loses information about the source structure being mapped. The union is an accurate type tho as this is what the callback would receive during traversal.

The current TypeBox Mapped implementation was able to find a way around the union parameter by encoding mappable keys in TMappedKey (as you saw), however I have only been able to get it to work for a linear set of keys / properties. Supporting deeply recursive structures would be another level of sophistication ... but not sure if it would be possible without TypeScript support (specifically someway to reconcile callback implementations (JS) to higher kinded types (TS)), or have TS able to derive types from implementations.

// from a constrained parameter T
function Test<T extends 1 | 2 | 3>(value: T): computed { // < new language feature?
  // ... and from this control flow
  return (
    value === 1 ? "A" :
    value === 2 ? "B" :
    value === 3 ? "C" :
    (() => { throw 1 })()
  )
}

// ... ts should be able to derive this type  (for computed)
type TTest<T extends 1 | 2 | 3> = (
  T extends 1 ? "A" :
  T extends 2 ? "B" :
  T extends 3 ? "C" :
  never
)

Maybe one day


Anyway, thank you for taking the time to submit the Visit functionality. I'll put a review tag on this issue in case a future TS language feature lands to make it possible to revisit this functionality. Being able to map TB types through function implementations while retaining TS types is a super sought after feature for this library (I can think of several hundred uses for it!) :D

All the best! S