JSON schema validation middleware

[arrterian]

Hi, guys :hand: I want to implement json-schema-validator for marblejs.

Why

Have written json-schema validation middleware for my pet project, and I wish to contribute to marblejs and thing that the solution can be useful for the marblejs community.

Why Joi middleware is not enough? Joi looks pretty good but I need to generate swagger specs for my API. So use the same schema for validation and documentation is a good idea in my opinion.

Internal realization In my project I use typescript-json-schema for generation schema from typescript interfaces. So I have custom logic for schema loading and not sure that such solution will be good for marble.

Implementation proposal

I propose one of three ways implementation json-schema validator for marble.

Use typescript-json-schema API for generating schema from typescript types or interfaces

The way is fully based on typescript interfaces or types. So we have a simple described type and generate JSON schema in runtime (when application start). Then we keep generated schemas in memory and use for validation or/and serving swagger specs (if we need).

Prebuild schemas for validation

This way is similar to previous, but instead compiling JSON schema on the fly we will build schemas before and just read the schemas from files.

Pass JSON schema manually

This way is simplest from this list. Just pass JSON schema as a simple object (and marble users should implement source yourself). It will like as Joi middleware, but instead passing Joi builder we will pass JSON schema object.

I think last way it preferred for marble, such as not all users need the functionality for generation schemas and serving specs.

Questions

1. What you guys think about implementing such middleware?
2. If you approve implementing the middleware which way you prefer?
3. Should I clarify something from the description above?

[krzysztof-miemiec]

Thank you for taking your time and describing the issue precisely! 💪 I personally think that we could generate the schema basing on the current Joi middleware, but I wouldn't mind having a separate json-schema-middleware. While briefly drafting our next release, we've discussed idea of creating utility that will generate documentation basing on tests 👉 you write & run tests = you get docs for free. Also, there is a work in progress on proper Joi middleware typings. We have to think about some concepts regarding the utility, but in the end, it should provide some raw JSON request-response pairs which can be then presented as a swagger doc, Apiary api blueprint or something else.

Having documentation ready on server launch sounds great, but how do you show server responses? 🤔 Would you use TypeScript's types for responses too? Our reasoning behind having tests->docs is that you can collect the variety of responses (ranging from a couple of 200s with different requests/responses to 4xxs) and cover all cases, presenting them in a nicely combined manner.

[arrterian]

@krzysztof-miemiec Now docs not generating fully automatically. I just use one schema source for validation and swagger specs. Swagger specs at the time writing by hand.

I have the following flow for validation now.

1. All types that produce JSON schema have suffix .schema.ts
2. All .schema.ts files have one class inside. The class have only dummy properties and using only as a data structure. I tried to use type keyword but such a thing isn't available in JS runtime, so I use classes as simple data structure now.

Example

export class UserSchema {
  firstName: string
  lastName: string

  /**
   * @format email
   */
  email: string

  /**
   * @format datetime
   */
  createdTime: string
}

3. Gulp task process files by pattern .schema.ts and create files .schema.json in folder that contains all project json schemas.

Example

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "properties": {
        "createdTime": {
            "format": "datetime",
            "type": "string"
        },
        "email": {
            "format": "email",
            "type": "string"
        },
        "firstName": {
            "type": "string"
        },
        "lastName": {
            "type": "string"
        }
    },
    "type": "object"
}

4. After in validation file I imports the class and pass as function argument to validator.

// interface
const validate<T>(schema: T, data: any) => T

So, we can use passed class for type inference our data and such as schema is a class available in runtime, we can use schema.name for reading our json from the folder and validate the data.

[arrterian]

@krzysztof-miemiec BTW, your way with generating response example from tests is awesome!

[JozefFlakus]

Hi Roman!

I like the idea! The more available middlewares and solutions, the more it can be interesting for end users. The second solution seems to be more elegant and especially useful when we will introduce the improved type inference, that you mentioned here #78.

As @krzysztof-miemiec mentioned, we are currently working on this thing. The core mechanism for inferring type from composed middleware is done, but not yet released. We want to release it together with improved @marblejs/middleware-joi which after all should be able to infer types based on provided Joi.Schema objects.

[arrterian]

@JozefFlakus Okay) Then I think on this week I can start implementing the middleware 🙂

[JozefFlakus]

Regarding the json-schema definition I'm not sure which way could be better in the end. The types/interfaces are more reusable and the ways of possible definitions are closed but on the other hand with classes we can think also about decorators, which can open much more possibilities, but still... these are classes 🤨

[arrterian]

@JozefFlakus With classes, we can pass the class as a parameter and use for runtime, when with generic parameter we should always pass schema name as a string or json-schema-object and desired type as generic directly. I also not fun classes, but I think it's a good solution in this case. Also, I plan to make a private constructor for such classes, that it allows use it only as type.

[arrterian]

It sad, but TypeScript still is so poor for functional programming

[JozefFlakus]

@RomanValihura regarding my previous comment, I misunderstood the point... 🙈At the end I would like to avoid generating build artifacts during the app bootstrapping, so the first solution seems to be more appropriate.

[arrterian]

@JozefFlakus I think it's reasonable. Cause on pre generating we need some like CLI interface for allowing generate a schema for marble users. And the second option will more convenient for use. And artifacts for using middleware component is too much.

[JozefFlakus]

@RomanValihura any updates in this topic? 😊

[arrterian]

@JozefFlakus Sad, but no update, yet. Hope I find a bit time on the weekend.

[JozefFlakus]

Closing due to inactivity.