Stranger6667 / jsonschema

A high-performance JSON Schema validator for Rust
https://docs.rs/jsonschema
MIT License
528 stars 94 forks source link

Validating individual definitions #432

Open sztomi opened 1 year ago

sztomi commented 1 year ago

I'm trying to validate JSON snippets against the DAP JSON schema: https://microsoft.github.io/debug-adapter-protocol/debugAdapterProtocol.json

Each type of message is a separate definition. I can extract and compile the relevant definition, but references are not resolved. I think I need to use the with_document function, but I'm not sure what the ID is supposed to be.

fn main() {
    let schema = include_str!("../debugAdapterProtocol.json");
    let schema: Value = serde_json::from_str(schema).unwrap();

    // Get the specific definition from the schema
    let next_request_schema = schema.get("definitions").unwrap().get("NextRequest").unwrap().clone();

    // Compile this specific schema
    let compiled = JSONSchema::options()
        .should_validate_formats(true)
        .should_ignore_unknown_formats(false)
        .with_document("#".to_string(), schema)
        .compile(&next_request_schema)
        .unwrap();

    let instance = json!({
        "seq": 153,
        "type": "request",
        "command": "next",
        "arguments": {
        "threadId": 3
        }
    });
    let result = compiled.validate(&instance);
    if let Err(errors) = result {
        for error in errors {
            println!("Validation error: {}", error);
            println!("Instance path: {}", error.instance_path);
        }
    }
}

This results in

Validation error: Invalid reference: json-schema:///#/definitions/Request
Instance path:
Validation error: Invalid reference: json-schema:///#/definitions/NextArguments
Instance path:

I tried an empty string as well, but that didn't work either. Any clues?

VoltaireNoir commented 9 months ago

I am facing the same problem with internal references. Were you able to figure it out?

sztomi commented 9 months ago

@VoltaireNoir with a very ugly hack: https://github.com/sztomi/dap-rs/blob/main/integration_tests/src/lib.rs#L27

VoltaireNoir commented 9 months ago

@VoltaireNoir with a very ugly hack: https://github.com/sztomi/dap-rs/blob/main/integration_tests/src/lib.rs#L27

Ah, I arrived at the same solution: to replace the references with the appropriate schema. I guess I'll stick with it until jsonschema is able to resolve local/internal references correctly.

eirnym commented 9 months ago

I have complex schema with dependencies between files and defined objects. I can't extract an object like @sztomi has shown in his example. Is it still possible to use this subschema to validate?

VoltaireNoir commented 9 months ago

I have complex schema with dependencies between files and defined objects. I can't extract an object like @sztomi has shown in his example. Is it still possible to use this subschema to validate?

I believe this library supports resolving web and file based references. Try enabling the resolve-http and resolve-file feature flags.

eirnym commented 9 months ago

Try enabling the resolve-http and resolve-file feature flags.

This solves a small portion of my goal.

I have several files common.yml, schema1.yml and schema2.yml (json schema is saved in safe yaml without references), all of these files contain all objects under $defs. Main schema contains only schema version, it's $id and minimal description. My goal is to create a validator object to validate by a definition for an object defined as above. Obviously loader is restricted by a folder where schemas are located and other locations are deliberately disabled to avoid security issues.

In current setup I use another validator (as it's a Python project) where I able to load given files into registry and then during application startup add referencing schemas to create an actual validator. Referencing schemas are anonymous schemas with references to an actual object defined in the registry, e.g. containing only $ref pointing to schema1.yml:#/$defs/obj1 or urn:schema:2:id:#/$defs/obj2 (where urn:schema:2:id is an $id value from schema2.yml file).

Could you please help me how to do a similar thing using this library?

thomas-burke commented 2 weeks ago

This is a very useful crate. Nicely done. I've worked around the limitation in a similar use case by iterating through the definitions and concatenating an "allOf:" [{"$ref":"#/definitions/<schema-needed-later>"}] to the end and then compiling the full schema for the definitions I need later.

example:

This is inefficient from a compile time & memory perspective, so I'd like to +1 this request.

Options I can think of would be:

  1. Add a way to simulate the "allOf" at validate-time (e.g. a new signature validator.validate(&json, "#/definitions/Event") or similar).
    • Overall this seems to be the most efficient way because it is compile-once, validate-many
  2. Add a way to compile the targeted schema schema without the need to do JSON manipulation (e.g. a new signature validator = validation_options.build(&schema3, "#/definitions/Event") or similar)
    • This would save the need for transient memory allocation that I do prior to build, but the compile-time and final memory costs would be similar to my current work-around.

I tried to implement a facsimile of (1) by making a version of validate that also accepts a LazyLocation, but I'm wasn't familiar enough with the code to make it work.

I think (1) would also help with this issue: https://github.com/Stranger6667/jsonschema/issues/452