qcho commented 5 years ago

I'm working on a project that will be heavily enhanced once the next JSON-Schema draft is released.

Since the next JSON-Schema draft is in released now as Draft 2019-09. I'm wondering if there are any efforts done now to make avj compatible as soon as possible or any plan at all for that. I didn't find anything in issues/pull-requests.

I've seen version 7.0.0 milestone but didn't find any official information about it

Thanks in advance!

More info https://json-schema.org/work-in-progress

epoberezkin commented 5 years ago

What draft-08 features are you interested in?

qcho commented 5 years ago

The most important I need is ref delegation instead of using avj's custom $merge...source...with. This will allow doing things such as defining default values alongside $ref

{
    "title": "Feature list",
    "type": "array",
        "items": [
            {
                "title": "Feature A",
                "properties": {
                    "enabled": {
                        "$ref": "#/$defs/enabledToggle",
                        "default": true
                    }
                }
            },
            {
                "title": "Feature B",
                "properties": {
                    "enabled": {
                        "description": "If set to null, Feature B
                                        inherits the enabled
                                        value from Feature A",
                        "$ref": "#/$defs/enabledToggle"
                    }
                }
            }
        ]
    },
    "$defs": {
        "enabledToggle": {
            "title": "Enabled",
            "description": "Whether the feature is enabled (true),
                            disabled (false), or under
                            automatic control (null)",
            "type": ["boolean", "null"],
            "default": null
        }
    }
}

example from rfc.section.7.6.1.1

Second $vocabulary

Also there will be some changes in the syntax such as definitons => $defs that would make using avj useless in draft-8.

Draft-8 milestone: https://github.com/json-schema-org/json-schema-spec/milestone/6

qcho commented 5 years ago

The Appendix B. ChangeLog is very useful to see what needs to be done.

draft-handrews-json-schema-WIP

        Update to RFC 8259 for JSON specification
        Moved "definitions" from the Validation specification here as "$defs"
        Moved applicator keywords from the Validation specification as their own vocabulary
        Moved the schema form of "dependencies" from the Validation specification as "dependentSchemas"
        Formalized annotation collection
        Specified recommended output formats
        Defined keyword interactions in terms of annotation and assertion results
        Added "unevaluatedProperties" and "unevaluatedItems"
        Define "$ref" behavior in terms of the assertion, applicator, and annotation model
        Allow keywords adjacent to "$ref"
        Note undefined behavior for "$ref" targets involving unknown keywords
        Add recursive referencing, primarily for meta-schema extension
        Add the concept of formal vocabularies, and how they can be recognized through meta-schemas
        Additional guidance on initial base URIs beyond network retrieval
        Allow "schema" media type parameter for "application/schema+json"
        Better explanation of media type parameters and the HTTP Accept header
...

epoberezkin commented 5 years ago

Allowing additional keywords alongside $ref is supported - there is an option.

definitons vs $defs only makes difference from the perspective of validating against meta-schema - references would work in the same way.

On other items in the list I will consider once it's published.

qzb commented 5 years ago

@epoberezkin It was published two days ago.

handrews commented 4 years ago

Also there will be some changes in the syntax such as definitons => $defs that would make using avj useless in draft-8.

That really shouldn't be a problem- definitions is still reserved for compatibility by the default meta-schema, and since the keyword (under either name) doesn't actually do anything, it should work unless an implementation specifically does something to prevent it. This keyword is just a reserved location to put schemas- you could put them anywhere, but here the meta-schema will help you recognize that it's actually supposed to be a schema instead of just kind of hoping that it works (there are potential subtle bugs to "just kind of hoping that it works" but they're pretty obscure and involve some really weird usage before you trigger them).

neverendingqs commented 4 years ago

From the docs:

var ajv = new Ajv({schemaId: 'id'});
// If you want to use both draft-04 and draft-06/07 schemas:
// var ajv = new Ajv({schemaId: 'auto'});
ajv.addMetaSchema(require('ajv/lib/refs/json-schema-draft-04.json'));

Should I add require(ajv/lib/refs/json-schema-draft-07.json) in lieu of there not being a 2019-09 version?

EDIT: sorry it looks like I should be able to proceed without referencing the draft if I am using the latest version.

EDIT:

If my schema contains

  "$schema": "https://json-schema.org/draft/2019-09/schema#",

ajv errors out with

Error: no schema with key or ref "https://json-schema.org/draft/2019-09/schema#"

epoberezkin commented 4 years ago

You either have to use draft-7 in your $schema or you can add new metaschema file to your app and explicitly add it. It won’t make new keywords supported of course.

nnmrts commented 4 years ago

@epoberezkin What's the current status on this? Can I help somehow? I (and probably many others) would love to have 2019-09 support.

And I'd like to help but I really don't understand the dot syntax and/or this project's structure. Which is obviously my fault. But still.

So, my real question is: Are you planning to add support for 2019-09?

epoberezkin commented 4 years ago

@nnmrts Unfortunately, 2019-09 is very difficult to consistently implement. My plan is to continue supporting draft-07 of the specification, to skip the proposed draft 2019-09 and to consider implementing the next draft, if some decisions of the proposed 2019-09 draft are revisited and radically simplified. cc @Relequestual - happy to work with the draft specification maintainers towards this goal.

As a side comment, it would be great to see JSON Schema evolving towards RFC - it would require some simplification of draft-07 and, possibly, some simpler solutions to the limitations that the proposed draft 2019-09 tried to address. Unfortunately, 2019-09 is the step away from RFC.

handrews commented 4 years ago

And yet OpenAPI has adopted draft 2019-09 for OAS 3.1. The JSON Schema ecosystem will continue to move forward, and re-unifying with OpenAPI means this will be the most broadly adopted draft since draft-04.

epoberezkin commented 4 years ago

@nnmrts doT syntax is actually very simple and as a light-weight approach to code generation is much easier to maintain than AST would have been.

There are some valuable improvements to Ajv you could have contributed - it would be very helpful.

zdne commented 4 years ago

@epoberezkin

First of all, thanks for all the amazing work you have done on AJV! 🙇‍♂️

@nnmrts Unfortunately, 2019-09 is very difficult to consistently implement. My plan is to continue supporting draft-07 of the specification, to skip the proposed draft 2019-09 and to consider implementing the next draft, if some decisions of the proposed 2019-09 draft are revisited and radically simplified.

Do I read this correctly that the draft 2019-09 support is not likely in AJV?

nnmrts commented 4 years ago

@epoberezkin While I do understand implementing 2019-09 is difficult, I would suggest to at least work on a "lite" package, implementing all the incompatible changes and semi-incompatible changes for now.

I'd even be fine with just the semi-incompatible changes, because the current version of ajv breaks when using $defs instead of definitions even though it's the exact same thing.

All the other changes are rarely used in my opinion and could be implemented later on.

Relequestual commented 4 years ago

@epoberezkin The utility of JSON Schema has gone beyond the original concept, and it now has far more utility and extensibility, required to take it to the next level.

This move has been justified, and now confirmed as the right direction, by the acceptance of draft 2019-09 by the OpenAPI specification, the largest and most well used API specification format.

Simply, totally revisiting the new concepts introduced by draft 2019-09 is not only a direction we don't want to go down, but is now also unfeasable given OpenAPI 3.1 support.

It IS possible that you could define a "vocabulary" which constains the format to a limited subset of keywords, but you would still need to support vocabularies in the first place. I expect there might be such a thing for non-relational, single schema level, schema to class code generation, at some point.

2019-09 is very difficult to consistently implement

I'd be interested to hear what problems you've been having when trying to implement. Maybe other implementers of 2019-09 can help you re-think your approach.

Relequestual commented 4 years ago

I've mentioned previously, I'm more than happy to discuss and talk through how vocabularies works. I know there's a thread we have somewhere where you expressed a number of concerns, which I believe can be alleviated by understanding, but I haven't had the time to follow up. For that I'm sorry. My door is open.

nnmrts commented 4 years ago

I know this maybe sounds annoying, but 2019-09 is out for more than 3 months now and it seems like @epoberezkin is just not planning to implement it.

As I said previously, I just don't get the doT syntax, I don't get the project structure, I don't know why there is need for a template engine in a JSON Schema validator and I don't feel like learning yet another opinionated language with outdated docs.

And yeah, maybe this sounds rude, but answering that you will only support the newest release of the very thing you advertise to validate if a specific condition is met ("and to consider implementing the next draft, if some decisions of the proposed 2019-09 draft are revisited and radically simplified") and only if the authors of that specific thing change their mind about how it should look like, then that's sounds rude to me too.

I don't want to make mountains out of molehills, but people's work actually depends on this project. GitHub's "Used by"-feature shows 4.7m, who knows how many projects actually use it. And who knows how many people use JSON Schema (maybe you, @Relequestual), but have to switch to .NET or something like that because their favorite language doesn't support the newer version. ajv is one of the most popular implementations. And sadly also one of the very few implementations, where the author explicitly said they don't plan to support 2019-09.

Well, okay, where does this lead us? Why did I even write this?

2 Questions:

Is there a workaround to this problem without switching programming languages?
Would it be a good idea to start a new validator project with 2019-09 support? Because I'm actually planning to do it. (Please prevent me from doing this...)

Thanks.

epoberezkin commented 4 years ago

@nnmrts I completely understand that you want to use JSON Schema, and you [justifiably] believe that its latest draft is the best. I do strongly disagree with it though. I believe the best draft to use at the moment is draft-07, and I believe that Open API would make a mistake if they proceed with the adoption of 2019-09, that is experimental, not backwards compatible and very difficult to implement, and not of draft-07, that is stable and widely supported by many validators across multiple languages. Tagging @darrelmiller @earth2marsh @MikeRalphson @webron @whitlockjc for visibility of this conversation.

I will provide a bit more detailed feedback a bit later to everybody about proposed draft 2019-09 and what is missing there to enable its support and what makes it so difficult to implement.

Not supporting the latest proposed draft is not a problem as such, that I or anybody have to solve. You can perfectly use draft-07 schemas for majority of validation scenarios people need. A lot of JSON Schema validators do not even support draft-07 in full, and even they can be used for majority of scenarios.

Ajv took more than 4 years of development, starting from draft 4, skipping draft 5 that was ambiguous, then supporting draft 6 where the problems were fixed, and then supporting draft 7. I've both contributed to Ajv and to JSON Schema evolution itself over this time. That Ajv became the most compliant with the specification via miticulously fixing all reported bugs, and also its adoption by some widely used libraries (e.g. request, webpack, eslint), led to its widespread adoption - which is both pride and a sense of obligation to the community of Ajv users.

If I implement draft 2019-09, I will have to maintain the same quality standards, and achieving it requires substantial time investment - probably 1-3 months full time work - it's very difficult to estimate at this point. And it's just to release it. It will take further 6-12 months of active development to stabilise it. So while it is possible, it is prohibitively expensive - the proposed paradigm shift in validation is quite substantial.

So, whether 2019-09 or any further drafts are supported is a question yet to answer - but I am under no obligation to do it. Also, I was very transparent to JSON Schema maintainers, both @handrews and @Relequestual, that the support of 2019-09 in Ajv will be very difficult and unlikely, due to the validation paradigm change that is not only difficult to implement, but also, I believe, can be very confusing to the end users. I provided this feedback continuously, during the course of the preparation on this draft. Unfortunately, my comments were not taken into account.

Also, JSON Schema is not a standard yet. It is a draft of a future possible standard. So supporting and using draft 2019-09 is not mandatory. I believe that the community of JSON Schema users should continue using draft-07 until the next versions becomes stable, the ambiguities are clarified and the missing test cases are added.

The alternative option would be splitting the specification. There are four different kinds of XML schemas, so I am not sure why there should be only one JSON schema - there can be alternatives. The direction proposed in https://github.com/json-schema-org/json-schema-spec/issues/710 by @ucarion makes more sense to me than the direction that 2019-09 took.

epoberezkin commented 4 years ago

@nnmrts answering specific questions you asked:

I don't know why there is need for a template engine in a JSON Schema validator and I don't feel like learning yet another opinionated language with outdated docs.

Ajv is a "compiling" validator. It treats JSON Schema as DSL and compiles it to JavaScript functions (closures). It is the only way to achieve server side validation performance, interpreting JSON Schema during validation can be 100 times slower.

You can generate code using AST (babel) or templates, templates seemed an easier choice. So doT templates you are talking about actually render JavaScript code. There has been several contributors who did substantial changes and contributions to it - so it is definitely possible to figure out.

Is there a workaround to this problem without switching programming languages?

You can deploy .NET library as a service, but I cannot judge how comprehensive it is.

Would it be a good idea to start a new validator project with 2019-09 support? Because I'm actually planning to do it. (Please prevent me from doing this...)

I would be very happy to see another JSON schema validator that I can borrow ideas from - so please do it.

handrews commented 4 years ago

@epoberezkin Draft 2019-09 was intentionally written in such a way that implementations that wished to focus only on validation and ignore features such as annotation collection should do so.

This was done specifically as a result of your feedback, to ensure that dedicated validation implementations (the vast majority of JSON Schema projects currently in production, at least outside of OpenAPI) were not forced to change their entire architecture in order to expand into new use cases.

In particular, Section 7.3 Default Behaviors makes this clear:

Because annotation collection can add significant cost in terms of both computation and memory, implementations MAY opt out of this feature. Keywords known to an implementation to have assertion or applicator behavior that depend on annotation results MUST then be treated as errors, unless an alternate implementation producing the same behavior is available. Keywords of this sort SHOULD describe reasonable alternate approaches when appropriate. This approach is demonstrated by the "additionalItems" and "additionalProperties" keywords in this document.

Note that we specifically ensured that existing keywords additionalItems and additionalProperties, despite being rephrased in terms of annotation collection, are not required to be literally implemented in those terms. They can continue to be implemented in whatever way already works in any project.

Note also that the wording ensures that the two keywords that depend on annotation collection, unevaluatedItems and unevaluatedProperties, can either be treated as errors or implemented in whatever other way an implementation prefers, as long as the behavior is the same. This was done to avoid burdening existing implementations with complex features that might required deep changes. There is a great deal of demand for unevaluatedProperties in particular, but we made it essentially optional because of your feedback, despite that making the error-out behavior potentially surprising.

Furthermore, our recommendations for output formats are recommendations rather than requirements to avoid forcing existing implementations to change. For validation-only implementations that do wish to use a recommended format, we included a minimal validation-only format. But that is just an option, not a requirement.

The non-validation use cases are very common, particularly with OpenAPI but also numerous other projects such as various web form generators. We did our best to ensure that implementations such as Ajv that wish to optimize validation can continue to do so, while also providing ways to support the other use cases that have been common in practice for years, but poorly supported by the specification. I'll leave it to the @OAI/tsc to make their own response, but OpenAPI-specific tooling is largely concerned with non-validation use cases such as documentation and code generation.

As for project "forks," note that @ucarion has, with our encouragement, begun work on his proposal as a separate project with its own name, JSON Data Definition Format (JDDF).

darrelmiller commented 4 years ago

@epoberezkin For better or worse, a significant amount of JSON Schema usage in the OpenAPI world is not for validation. It is for driving code generators and creating documentation. It is used as a type description language. The distinction between annotations and validation keywords supported in 2019-09 draft enables what @qcho and many other of our users want to do, in a manageable way. In my opinion, this feature was one of the tipping points that drove us to do the work that would align OpenAPI schema and JSON Schema.

ucarion commented 4 years ago

It seems unreasonable to ask @epoberezkin to implement a specification without test cases. His freely-donated time is clearly limited, can we do something to make it easier to support this ask?

Coming up with a suite of unambiguous examples, in the form of tests, constitutes a big chunk of the work required to implement a specification. Moreover, this is work the I-D authors are in a better place than anyone else to do, since the JSON Schema authors have surely written some toy implementations to make sure the spec holds water.

@handrews @Relequestual I'm sure y'all have made implementations (toy or otherwise) of the latest draft. Can y'all share those, to help evaluate what it would take to port stuff over to ajv? That too could really help move this along.

epoberezkin commented 4 years ago

Thanks for the comments @handrews. As I wrote, I will comment in detail later the feedback from the spec review.

I’ve found JSL indeed, not quite sure why it had to be renamed to JDDF. “JSON schema“ is a generic term meaning “a schema for JSON data”. It cannot belong to any particular flavour of JSON schema - competition is a good thing.

@darrelmiller looking at the issues I had, a lot of people use schemas in Open API definition for request / response body validation, so I am not sure why you say that the validation is not the primary use case. I’ve added “nullable” support because it was requested several times. For code generation JSL/JDDF could be better as it is more aligned with type systems.

handrews commented 4 years ago

@ucarion we definitely do not expect wide adoption without test cases, and folks are working on that. @gregsdennis has a .NET implementation which he updated as we wrote the spec. We're using that to help produce the test suite, which he and @Julian are working on.

I don't think anyone is demanding that @epoberezkin implement support right this minute. There are other implementations working on 2019-09 support but I don't think anyone else is close to being done, in part because of the test suite not being in place. We'd welcome additional help on the test suite.

We're also expecting adoption to really ramp up once OpenAPI 3.1 is published.

The JSON Schema project doesn't expect/demand that anyone do anything, btw. We publish drafts and we hope they are interesting enough to pick up. OpenAPI and several other validator implementors have decided that 2019-09 is, while @epoberezkin has decided that it is not. That's all fine, this is open source after all.

Julian commented 4 years ago

Definite +1 on help welcome for the test suite.

philsturgeon commented 4 years ago

@epoberezkin OpenAPI is for many things. Validation is just one use case. Here are a whole pile of other use cases. http://OpenAPI.Tools definitely not just validation.

I’ve recently been trying to get people to understand it is possible to use it for request validation, because most people in fact do not use it for that. They are starting to, but the “validation is the primary use case” that you and @ucarion generally follow does not apply to OpenAPI.

philsturgeon commented 4 years ago

Also nobody is making any demands of anyone’s time. If y’all too busy to keep AJV as an active project I’m sure plenty of people would be happy to step in. This is why I usually advocate moving popular projects to organisations, to reduce bus factor and limit the requirements of a single persons time. Nobody wants an open source martyr, and nobody wants people working on features they don’t like, but that doesn’t mean a whole specification can be changed to go against consensus just to make that person like it more. 😅

epoberezkin commented 4 years ago

Also nobody is making any demands of anyone’s time.

@philsturgeon, unfortunately, it is not the case.

@nnmrts wrote above:

I know this maybe sounds annoying, but 2019-09 is out for more than 3 months now and it seems like @epoberezkin is just not planning to implement it. I don't want to make mountains out of molehills, but people's work actually depends on this project. GitHub's "Used by"-feature shows 4.7m, who knows how many projects actually use it. And who knows how many people use JSON Schema (maybe you, @Relequestual), but have to switch to .NET or something like that because their favorite language doesn't support the newer version. ajv is one of the most popular implementations. And sadly also one of the very few implementations, where the author explicitly said they don't plan to support 2019-09.

The way I read what @nnmrts wrote, is that because 2019-09 draft was proposed for review 3 months ago, and "ajv is one of the most popular implementations" (@nnmrts, thank you by the way), I should feel bad for not having implemented it yet.

To me it seems a very thinly veiled demand on my time to implement 2019-09. As does another comment, that @nnmrts thankfully retracted, but it landed in my mailbox anyway (and in anybody else's who follows this thread).

To clarify my position on support for the proposed draft 2019-09:

I did not yet make a decision whether 2019-09 will be supported, so saying I don't want or do not plan to implement it is incorrect.
I am actually open to implementing 2019-09. I voiced the concerns about the ambiguity for the end users, implementation complexity and costs, and the lack of test cases. These concerns are possibly addressable - by providing easy to follow tutorial for the full validation spec, sponsorship of this 2019-09 implementation in Ajv and adding the test cases. I also proposed an alternative way to address them - to simplify the specification and implement the next draft (I do hope there will be the next draft). I do not demand any simplification at all.
I have several specific concerns I promised to share about 2019-09, and I will - please be patient, it is taking some time to carefully review the specification and to summarise all the points. Even if the community of maintainers disagrees with my feedback, it would still be very helpful to the end users to decide what of the available validation specifications to use for their validation scenarios.

This is why I usually advocate moving popular projects to organisations, to reduce bus factor and limit the requirements of a single persons time.

@philsturgeon I actually considered moving Ajv and related packages to the organisation (@ajv-validator) - it is likely to happen at some point this year.

I would appreciate if going forward we could limit this conversation to the merits of the proposed JSON Schema draft 2019-09, as it is really the main thing that has bearing on my decision to implement it. Whether I have resources or plans to implement it is secondary.

I am going to block this thread until the feedback to 2019-09 is prepared (it will take 2-4 weeks) and if you have any comments please reach out directly via email.

epoberezkin commented 4 years ago

I have reviewed 2019-09 draft.

The main sources of implementation complexity and uncertainty:

dynamic recursive references ($recursiveRef, $recursiveAnchor). This new feature substantially complicates the previous referencing model that was lexical only and, most likely, requires full rewrite of the reference resolution part of Ajv. It has to be investigated how dynamic references should be supported in compiled validation functions, but it should be possible. Rewriting $ref resolution part should also help addressing the existing limitations (e.g. #512, #815, #976, better support of $refs in serialisable validation functions created with ajv-pack, etc.). The explanations in the spec are not very straightforward and may require some clarifications and test cases (there seem to be no test cases for dynamic recursive references in JSON-Schema-Test-Suite)
unevaluatedProperties and unevalutatedItems. Annotation collection will have to be added to support them. It most likely will require additional test cases to test interaction with other applicator keywords, in addition to a couple of test cases in JSON-Schema-Test-Suite. This also will likely require substantial rewrite to avoid short-circuiting validation as required.
supporting multiple error formats.

Although implementing full support of 2019-19 in Ajv will be quite time consuming, it is possible. It will require substantial refactoring/rewrite - it could be the opportunity to migrate Ajv code-base to TypeScript and to make it more modular, maintainable and extensible.

A very rough estimate for this work is 3 months full time development, so it would require sponsorship. I am going to kick off a campaign for Ajv sponsorship - once the balance target (combined in GitHub and Open Collective) is reached, the support for 2019-09 (or the next one) will be implemented.

Also, I think it is sensible to prepare the additional test cases and to clarify the spec. Realistically, the support of the current (or next) draft could be ready by the end of 2020 if the sponsorship campaign is successful within a couple of months.

philsturgeon commented 4 years ago

@epoberezkin fantastic news! When you say successful, what sort of monthly amount do you need to get to (between GitHub and OC) for this work to happen?

epoberezkin commented 4 years ago

@philsturgeon I've put the target of $25k balance in https://opencollective.com/ajv. I'll start sooner than that of course. I won't do it on my own - I'll get some help. GitHub does not show the balance unfortunately, but there are about $240 at the moment (including their matching fund) - not too much. But I wasn't trying to get any sponsorship before, so let's see if it works.

epoberezkin commented 4 years ago

I am going to close it and open another ticket with the TODO list for ajv v7, including support of 2019-09

epoberezkin commented 4 years ago

1198

epoberezkin commented 4 years ago

Please comment on the questions here: https://github.com/ajv-validator/ajv/issues/1247#issuecomment-660940093

1247 is the scope for the first release of v7-alpha.

ajv-validator / ajv

Support JSON-Schema Draft 2019-09 (formerly known as draft-08) #1082

1198

1247 is the scope for the first release of v7-alpha.