mrlubos
commented
2 weeks ago Hey @dev-dsp, this has been requested a few times already. My only question is, how would you resolve something like self-referencing schemas?
Open dev-dsp opened 2 weeks ago
mrlubos
commented
2 weeks ago Hey @dev-dsp, this has been requested a few times already. My only question is, how would you resolve something like self-referencing schemas?
dev-dsp
commented
2 weeks ago @mrlubos, it's great to hear there’s been interest in this feature! I only found #993, which addresses a different aspect of schema references.
One way to manage recursive references could be a maxDepth option in the configuration. This would allow users to specify how many levels deep the references should expand. Once the set depth is reached, any further $refs would remain as references rather than unfolding indefinitely.
For instance, a configuration might look like this:
// Config option to limit unfolding depth
{
unfoldRefs: {
enabled: true,
maxDepth: 2 // Stops unfolding after 2 levels
}
}Consider a self-referencing schema structure like NodeSchema:
const NodeSchema = {
properties: {
value: { type: 'string' },
children: {
type: 'array',
items: { $ref: '#/components/schemas/Node' }
}
}
}With maxDepth: 2, this would unfold references up to two levels, beyond which $ref is left intact to prevent infinite loops:
// Schema after applying maxDepth = 2
const UnfoldedNodeSchema = {
properties: {
value: { type: 'string' },
children: {
type: 'array',
items: {
properties: {
value: { type: 'string' },
children: {
type: 'array',
items: {
properties: {
value: { type: 'string' },
children: {
type: 'array',
items: { $ref: '#/components/schemas/Node' } // Stops unfolding here
}
}
}
}
}
}
}
}
}@mrlubos Another approach could be providing helper utilities that resolve references on-demand. Here are a few possible patterns:
// 1. Static helper function in generated code:
export function resolveFilterSchema(filter: {type: string}) {
const mapping = {
'first': FirstFilterSchema,
'last': LastFilterSchema,
'regex': RegexFilterSchema,
// etc
} as const;
return mapping[filter.type as keyof typeof mapping];
}
// Ideal for direct lookups and runtime type validation without unfolding references,
// suitable for most non-recursive references.
// 2. Instance method on schema objects:
export const FilterRuleSchema = {
// ... existing schema ...
resolveFilter(type: string) {
return this.filter.discriminator.mapping[type];
}
} as const;
// This is useful when FilterRuleSchema instances might dynamically need to resolve
// types based on runtime information, maintaining separation from hard-coded mappings.
// 3. Getter property that provides resolved versions:
export const FilterRuleSchema = {
// ... base schema ...
get resolvedOneOf() {
return this.filter.oneOf.map(ref => {
const schemaName = ref.$ref.split('/').pop();
return schemas[schemaName]; // Where schemas is an export of all definitions
});
}
} as const;
// Suitable when references need to be resolved “just-in-time” for usage, offering
// efficient memory use by resolving only as necessary. This fits scenarios with frequent
// or varied lookups without statically embedding all schema data.
// 4. Type-safe schema registry pattern:
export const SchemaRegistry = {
schemas: {
FilterRule: FilterRuleSchema,
FirstFilter: FirstFilterSchema,
// etc
},
resolve<T extends keyof typeof this.schemas>(ref: string): typeof this.schemas[T] {
const schemaName = ref.split('/').pop() as T;
return this.schemas[schemaName];
}
} as const;
// This registry pattern provides high reusability and scalability, allowing for robust,
// centralized schema management. It’s type-safe and ideal for projects with numerous
// schema references that benefit from centralized access control.Oh, @mrlubos, and a different take that might work - how about using a Proxy to lazy-load the $refs? Instead of expanding everything up-front or on-demand via functions, we could wrap the schema in a Proxy and intercept access to any $ref. So when a $ref is actually accessed, the Proxy fetches and resolves it right then and there, caching the resolved schema for future access.
const schemas = {
FilterRule: {
properties: {
type: { type: 'string', enum: ['filter'], const: 'filter', title: 'Type' },
filter: {
oneOf: [
{ $ref: '#/components/schemas/RegexFilter' },
{ $ref: '#/components/schemas/RangeFilter' },
// more filters
],
title: 'Filter',
description: 'Filter configuration'
},
target_field: { type: 'string', title: 'Target Field', description: 'Target field' }
},
type: 'object',
required: ['type', 'filter', 'target_field']
},
RegexFilter: { /* RegexFilter schema details */ },
RangeFilter: { /* RangeFilter schema details */ }
};
// Proxy handler to resolve `$ref`s when accessed
const refResolverHandler = {
get(target, prop) {
if (target[prop] && target[prop].$ref) {
const ref = target[prop].$ref.split('/').pop();
if (schemas[ref]) {
target[prop] = new Proxy(schemas[ref], refResolverHandler); // Cache resolved schema
}
}
return target[prop];
}
};
// Wrap FilterRuleSchema in the Proxy
const FilterRuleSchema = new Proxy(schemas.FilterRule, refResolverHandler);
// Usage example
console.log(FilterRuleSchema.properties.filter.oneOf[0]); // Triggers resolution of RegexFilter schemaHere's also some kind of a combined approach:
// Schema Registry with all definitions
const schemas = {
FilterRule: {
properties: {
type: { type: 'string', enum: ['filter'], const: 'filter', title: 'Type' },
filter: {
oneOf: [
{ $ref: '#/components/schemas/RegexFilter' },
{ $ref: '#/components/schemas/RangeFilter' },
// Other filters
],
title: 'Filter',
description: 'Filter configuration'
},
target_field: { type: 'string', title: 'Target Field', description: 'Target field' }
},
type: 'object',
required: ['type', 'filter', 'target_field']
},
RegexFilter: { /* RegexFilter schema */ },
RangeFilter: { /* RangeFilter schema */ },
// Additional schemas
};
// Dynamic Resolver Helper Function with Depth Control
function resolveSchema(schema, depth = 0, maxDepth = 2) {
if (depth > maxDepth) return schema; // Stop if max depth is reached
// Lazy-expand any `$ref` properties within the schema
if (schema.$ref) {
const schemaName = schema.$ref.split('/').pop();
return resolveSchema(schemas[schemaName], depth + 1, maxDepth);
}
// Recursively resolve properties or items if they're objects or arrays
const resolvedSchema = { ...schema };
for (const key in schema) {
if (typeof schema[key] === 'object') {
resolvedSchema[key] = resolveSchema(schema[key], depth + 1, maxDepth);
}
}
return resolvedSchema;
}
// Lazy Proxy Handler
const refResolverHandler = {
get(target, prop) {
if (target[prop] && target[prop].$ref) {
const schemaName = target[prop].$ref.split('/').pop();
target[prop] = new Proxy(resolveSchema(schemas[schemaName]), refResolverHandler);
}
return target[prop];
}
};
// Wrapping main schema with lazy proxy
const FilterRuleSchema = new Proxy(schemas.FilterRule, refResolverHandler);
// Usage examples
// Direct access, lazy loads only when properties accessed, within max depth
console.log(FilterRuleSchema.properties.filter.oneOf[0]); // Resolves RegexFilter lazily
// Explicit deep resolution if we need it
const fullyResolvedFilterRule = resolveSchema(FilterRuleSchema, 0, Infinity);
console.log(fullyResolvedFilterRule); // Fully expanded to any depth@dev-dsp What's your use case for schemas? Any of these could be a valid approach, depending on the objective...
dev-dsp
commented
2 weeks ago In my project, I use schemas to create dynamic forms for user input. These forms are based on the schemas, which helps ensure that the data users enter is valid before it’s sent to the backend (which is built with FastAPI). Validating data on the frontend like this helps keep everything consistent when it reaches the backend.
I get these schemas from two main places:
The issue came up when I tried to reuse components that were built to work with “prepared” schemas (schemas where all $ref links were already resolved on the backend) - when I switched to using openapi-ts-generated schemas, which still contain $ref references, my components couldn’t handle these unresolved references.
Ideally, I need a way to resolve $refs on the frontend as I use them.
mrlubos
commented
2 weeks ago @dev-dsp prior to this, were you maintaining the schemas manually? Do you have any code to share? What are you using to validate the form input?
dev-dsp
commented
2 weeks ago @mrlubos No, I use FastAPI with Pydantic. Let me provide a simplified example of what I'm working with:
from enum import Enum
from typing import Literal, Union
from pydantic import BaseModel, Field
class ValidatorType(str, Enum):
BASIC = "basic"
ADVANCED = "advanced"
class FieldType(str, Enum):
INPUT = "input" # Regular input field
COMPOSITE = "composite" # Field with nested validation
# Base validator config
class BaseValidator(BaseModel):
type: ValidatorType
class BasicValidator(BaseValidator):
type: Literal[ValidatorType.BASIC]
required: bool
class AdvancedValidator(BaseValidator):
type: Literal[ValidatorType.ADVANCED]
pattern: str
Validator = Union[BasicValidator, AdvancedValidator]
class BaseField(BaseModel):
type: FieldType
class InputField(BaseField):
type: Literal[FieldType.INPUT]
key: str
label: str
class CompositeField(BaseField):
type: Literal[FieldType.COMPOSITE]
validator: Validator = Field( # First level of $refs created here
discriminator="type",
description="Validation rules"
)
label: str
MyField = Union[InputField, CompositeField]
class FormConfig(BaseModel):
"""
When .model_json_schema() is called, we get nested $refs structure
that needs resolution either on backend or frontend side
"""
field: MyField = Field(
discriminator="type", # Another level of $refs
description="Field configuration"
)So for part of the models, I am resolving all $refs on the backend using jsonref:
jsonref.replace_refs(
FormConfig.model_json_schema(),
lazy_load=False,
proxies=False,
)... and send the result as a JSON object to the frontend (Svelte). There, I walk over the schema and configure switches/inputs/...
Previously, I was working with dynamically generated models that weren't part of the static API definition (thus not present in openapi.json, appearing as any in TypeScript). But now I'm trying to use my form generation/validation code with the schemas of the static (exposed via openapi) models that are generated by openapi-ts, and these still contain $refs.
I understand that OpenAPI reference resolution might be outside the scope of a code generator and probably belongs in a dedicated library. Still, having even basic (e.g. for those defined within the same schema) reference resolution capabilities would be quite useful for common use cases like this one. Ideally, the generated code could have a built-in helper, so we wouldn't need to implement this logic in every project separately.
Description
Given a schema like this:
The schemas.gen.ts contains the following declarations:
I'd like to have an option to include the referred definitions inside the oneOf/anyOf/...
If that's more like
won't fixissue - I'd be grateful for any workarounds..