ThomasAribart / json-schema-to-ts

Infer TS types from JSON schemas ๐Ÿ“
MIT License
1.44k stars 31 forks source link

Create query param object from query param list #82

Closed thomasheartman closed 2 years ago

thomasheartman commented 2 years ago

Hi! First off: thanks for all the effort in creating and maintaining this library!

I have a question that I can't seem to figure out an answer to regarding using this library with express and express-openapi, and that I was hoping you could shed some light on:

In short, I'm wondering if there is a way to create a record from query parameters via FromSchema?

A little more context: the OpenAPI spec requires you to type query parameters as a list of objects. These objects have a name, a type, description, and so forth, but it is a list of spec-like objects. Something like:

const params = [
    {
        name: 'paramName',
        schema: {
            type: 'string',
        },
        in: 'query',
    },
] as const;

For endpoints that accept JSON payloads, using FromSchema to generate types has gone very smoothly, but it seems to be a bit trickier with query parameters. Express types out the request as something like this:

        req: Request<unknown, unknown, unknown, QueryParamObject>,

where QueryParamObject is an object with query param names as keys and corresponding types.

My problem is that I'd like to convert the list of query parameters into something that FromSchema can parse, so that we get a nice generated type based on the schema, but I can't figure out how to do it.

The list is obviously not a schema in and of itself, but it's easy enough to map it so that it becomes the equivalent of a properties object on an object-type schema. However, I keep running into issues with readonly and type inference and I'm just not making any headway. Do you have any suggestions as to what I could do here?

Sorry if this doesn't actually apply to this library. I was not the one who set everything up for the project, so I'm not entirely sure how everything is wired together. I do know, though, that the FromSchema method does a lot of our type conversion and that the internals looked pretty intimidating. It may well be that this is outside of what this package is responsible for , but I thought I'd try this as my first port of call.

Thanks very much for any tips, tricks, and insights you can offer.

ThomasAribart commented 2 years ago

Hello @thomasheartman ๐Ÿ‘‹

Sure no problem ! I think one of the difficulty is that removing readonly from a readonly string[] is not enough to obtain string[] because some mutating methods (like pop or push) are removed from readonly arrays and not re-added by removing the readonly keyword. But you can stick with readonly arrays.

First create a ParseOpenApiParams generic type with correct type constraints:

import type { JSONSchema } from "json-schema-to-ts";

type OpenApiParam = {
  readonly name: string;
  readonly schema: JSONSchema;
  readonly in: "query" | "other"; // (I guess there probably are some other sources here)
};

type ParseOpenApiParams<P extends readonly OpenApiParam[]> = RecurseOnParams<P>;

Next, recurse on your array with the following syntax:

import { O, L } from "ts-toolbelt";

type RecurseOnParams<
  P extends readonly OpenApiParam[],
  R extends O.Object = {}
> = {
  continue: RecurseOnParams<
    L.Tail<P>,
    L.Head<P>["in"] extends "query"
    ? O.Merge<
        R,
        { [key in L.Head<P>["name"]]: FromSchema<L.Head<P>["schema"]> }
      >
    : R
  >;
  stop: R;
}[P extends readonly [OpenApiParam, ...OpenApiParam[]] ? "continue" : "stop"];

This should work like a charm:

const params = [
  {
    name: "paramName1",
    schema: {
      type: "string",
    },
    in: "query",
  },
  {
    name: "paramName2",
    schema: {
      type: "string",
    },
    in: "query",
  },
  {
    name: "paramName3",
    schema: {
      type: "string",
    },
    in: "other",
  },
] as const;

type parsed = ParseOpenApiParams<typeof params>;
// => { paramName1: string, paramName2: string }
thomasheartman commented 2 years ago

Oh,amazing; thanks! I'll give that a look on Monday, for sure ๐Ÿ˜ One other thing that seems to trip me up is if one of the schemas use the enum property, because you get a nested readonly array. Would your answer cover that too, do you think?

Thanks again for the quick and very detailed answer ๐Ÿ™

ThomasAribart commented 2 years ago

Doesn't look like it would be a problem. If it is, you can always use DeepWritable and DeepReadonly utility types:

type DeepWritable<T> = { -readonly [P in keyof T]: DeepWritable<T[P]> };

type DeepReadonly<T> = T extends O.Object
  ? { readonly [P in keyof T]: DeepReadonly<T[P]> }
  : T;

type OpenApiParam = {
  name: string;
  schema: JSONSchema;
  in: "query" | "other";
};

type ParseOpenApiParams<
  $P extends OpenApiParam[] | DeepReadonly<OpenApiParam[]>,
  // Make $P writable in a second argument
  // (Extract is needed as a work around to what seems to be a TS bug)
  P extends OpenApiParam[] = Extract<DeepWritable<$P>, OpenApiParam[]>
> = RecurseOnParams<P>;

type RecurseOnParams<P extends OpenApiParam[], R extends O.Object = {}> = {
  continue: RecurseOnParams<
    L.Tail<P>,
    L.Head<P>["in"] extends "query"
      ? O.Merge<
          R,
          { [key in L.Head<P>["name"]]: FromSchema<L.Head<P>["schema"]> }
        >
      : R
  >;
  stop: R;
}[P extends [OpenApiParam, ...OpenApiParam[]] ? "continue" : "stop"];

This worked for me:

const params = [
  {
    name: "paramName1",
    schema: {
      type: "string",
      enum: ["foo", "bar"],
    },
    in: "query",
  },
  {
    name: "paramName2",
    schema: {
      type: "string",
    },
    in: "query",
  },
  {
    name: "paramName3",
    schema: {
      type: "string",
    },
    in: "other",
  },
] as const;

type parsed = ParseOpenApiParams<typeof params>;
// => { paramName1: string, paramName2: string }
thomasheartman commented 2 years ago

You're right, the nested enums don't appear to be a problem at all ๐Ÿ˜„ However, I think I have found an actual issue with your solution and I'm a bit at a loss as to how to fix it: it appears the first item in the list becomes unknown:

Given this setup (simple copy-paste of the first snippet you provided):

import { FromSchema, JSONSchema } from 'json-schema-to-ts';

import { O, L } from 'ts-toolbelt';

type OpenApiParam = {
    readonly name: string;
    readonly schema: JSONSchema;
    readonly in: 'query' | 'other';
};

type ParseOpenApiParams<P extends readonly OpenApiParam[]> = RecurseOnParams<P>;

type RecurseOnParams<
    P extends readonly OpenApiParam[],
    R extends O.Object = {},
> = {
    continue: RecurseOnParams<
        L.Tail<P>,
        L.Head<P>['in'] extends 'query'
            ? O.Merge<
                  R,
                  {
                      [key in L.Head<P>['name']]: FromSchema<
                          L.Head<P>['schema']
                      >;
                  }
              >
            : R
    >;
    stop: R;
}[P extends readonly [OpenApiParam, ...OpenApiParam[]] ? 'continue' : 'stop'];

and this data:

const x = [
    {
        name: 'param1',
        schema: {
            type: 'string',
            enum: ['three', 'four'],
        },
        in: 'query',
    },
    {
        name: 'param2',
        schema: {
            type: 'string',
            enum: ['one', 'two'],
        },
        in: 'query',
    },
] as const;

I get this type:

type T = ParseOpenApiParams<typeof x>;

// type T = {
//     param1: unknown,
//     param2: "one" | "two"
// }

If I switch the order of the items in the list, param2 becomes unknown and param1 becomes "three" | "four". If the list contains only one item, it is typed as unknown. Even with more items in the list, this seems to only happen with the first item. This also occurs regardless of the type of the first param and whether it has enums.

I'm guessing that something happens the first time the items are split into [head, ...tail] that causes this, but I'm not able to diagnose it correctly ๐Ÿค”

ThomasAribart commented 2 years ago

Strange, I get the correct result:

Capture dโ€™eฬcran 2022-07-26 aฬ€ 21 11

Which version of ts-toolbelt / json-schema-to-ts are you using ? Mine were respectively 2.5.4 and 9.6.0. Can you also share your tsconfig.json ?

thomasheartman commented 2 years ago

Huh, interesting ๐Ÿค” It seems we were using (^)2.5.3 of json-schema-to-ts and 9.6.0 of ts-toolbelt. I tried updating it to 2.5.4, but it doesn't seem to make a difference. The code does work even without it, but that seems to by coincidence, so it's brittle.

Here's the tsconfig.json file

`tsconfig.json` ```json { "compilerOptions": { /* Visit https://aka.ms/tsconfig.json to read more about this file */ /* Basic Options */ // "incremental": true, /* Enable incremental compilation */ "target": "es2020" /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019', 'ES2020', or 'ESNEXT'. */, "module": "commonjs" /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', 'es2020', or 'ESNext'. */, "lib": [ "es6" ] /* Specify library files to be included in the compilation. */, "allowJs": true /* Allow javascript files to be compiled. */, // "checkJs": true, /* Report errors in .js files. */ // "jsx": "preserve", /* Specify JSX code generation: 'preserve', 'react-native', or 'react'. */ "declaration": true /* Generates corresponding '.d.ts' file. */, // "declarationMap": true, /* Generates a sourcemap for each corresponding '.d.ts' file. */ // "sourceMap": true, /* Generates corresponding '.map' file. */ // "outFile": "./", /* Concatenate and emit output to single file. */ "outDir": "./dist" /* Redirect output structure to the directory. */, "rootDir": "./src" /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */, // "composite": true, /* Enable project compilation */ // "tsBuildInfoFile": "./", /* Specify file to store incremental compilation information */ // "removeComments": true, /* Do not emit comments to output. */ // "noEmit": true, /* Do not emit outputs. */ // "importHelpers": true, /* Import emit helpers from 'tslib'. */ // "downlevelIteration": true, /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */ // "isolatedModules": true, /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */ /* Strict Type-Checking Options */ // "strict": true, /* Enable all strict type-checking options. */ // "noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */ // "strictNullChecks": true, /* Enable strict null checks. */ // "strictFunctionTypes": true, /* Enable strict checking of function types. */ // "strictBindCallApply": true, /* Enable strict 'bind', 'call', and 'apply' methods on functions. */ // "strictPropertyInitialization": true, /* Enable strict checking of property initialization in classes. */ // "noImplicitThis": true, /* Raise error on 'this' expressions with an implied 'any' type. */ // "alwaysStrict": true, /* Parse in strict mode and emit "use strict" for each source file. */ /* Additional Checks */ // "noUnusedLocals": true, /* Report errors on unused locals. */ // "noUnusedParameters": true, /* Report errors on unused parameters. */ // "noImplicitReturns": true, /* Report error when not all code paths in function return a value. */ // "noFallthroughCasesInSwitch": true, /* Report errors for fallthrough cases in switch statement. */ // "noUncheckedIndexedAccess": true, /* Include 'undefined' in index signature results */ /* Module Resolution Options */ // "moduleResolution": "node", /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */ "baseUrl": "./src" /* Base directory to resolve non-absolute module names. */, // "paths": {}, /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */ // "rootDirs": [], /* List of root folders whose combined content represents the structure of the project at runtime. */ // "typeRoots": [], /* List of folders to include type definitions from. */ // "allowSyntheticDefaultImports": true, /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */ "esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */, // "preserveSymlinks": true, /* Do not resolve the real path of symlinks. */ // "allowUmdGlobalAccess": true, /* Allow accessing UMD globals from modules. */ /* Source Map Options */ // "sourceRoot": "", /* Specify the location where debugger should locate TypeScript files instead of source locations. */ // "mapRoot": "", /* Specify the location where debugger should locate map files instead of generated locations. */ // "inlineSourceMap": true, /* Emit a single file with source maps instead of having a separate file. */ // "inlineSources": true, /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */ "sourceMap": true, /* Experimental Options */ // "experimentalDecorators": true, /* Enables experimental support for ES7 decorators. */ // "emitDecoratorMetadata": true, /* Enables experimental support for emitting type metadata for decorators. */ /* Advanced Options */ "resolveJsonModule": true /* Include modules imported with '.json' extension */, "skipLibCheck": true /* Skip type checking of declaration files. */, "forceConsistentCasingInFileNames": true /* Disallow inconsistently-cased references to the same file. */ }, "types": ["src/types/openapi.d.ts"], "exclude": [ "bin", "docs", "docker", "examples", "migrations", "node_modules", "website", "src/binver-dev.js", "dist", "snapshots", "coverage", "website", "scripts" ] } ```
ThomasAribart commented 2 years ago

Ok I managed to reproduce the bug. You miss the strict: true option recommended by ts-toolbelt: https://github.com/millsp/ts-toolbelt#prerequisites

This causes O.Merge to bug.

You can either set the strict option to true (which I recommend anyway, because it is good practice, but might require some migrations in your code), or not use O.Merge and use the & operator instead:

type RecurseOnParams<
  P extends readonly OpenApiParam[],
  R extends O.Object = {}
> = {
  continue: RecurseOnParams<
    L.Tail<P>,
    L.Head<P>["in"] extends "query"
      ? R & {
          [key in L.Head<P>["name"]]: FromSchema<L.Head<P>["schema"]>;
        }
      : R
  >;
  stop: A.Compute<R>; // You can use A.Compute to prettify the result
}[P extends readonly [OpenApiParam, ...OpenApiParam[]] ? "continue" : "stop"];

I hope it solved your pb ! Can I close the issue ?

thomasheartman commented 2 years ago

Oh, sick! Yeah, that explains it. Sadly, turning on strict mode for the entire project isn't an option at the moment (it'd just be too much work), but it's definitely something we want to do in the future. But if using & works too, I'll do that instead for now. I'll go ahead and make those changes when I'm back in work tomorrow.

And yeah, please feel free to close this issue! I'll reopen if it still doesn't work for some reason. Thanks so much for your help! It's super appreciated ๐Ÿ™๐Ÿผ ๐Ÿ’ฏ

thomasheartman commented 2 years ago

Just wanna chime in and confirm that using & and A.Compute works like a charm! Thanks again ๐Ÿ˜„