jam01 / json-schema

Apache License 2.0
3 stars 0 forks source link

Observations #2

Open Quafadas opened 6 days ago

Quafadas commented 6 days ago

I had a go at this, and bumped into the following, I needed to add these imports myself, in order to be able to run the example on the readme, could be helpful to add them to the docs.

import io.github.jam01.json_schema.*
import io.github.jam01.json_schema
import upickle.core.Visitor

The example on the homepage didn't work - it failed with linking errors.


[242] [info] compiling 1 Scala source to /Users/simon/Code/vecxt/out/jsSite/test/compile.dest/classes ...
[242] No such entry: plugin.properties
[242] [warn] No plugin in path /Users/simon/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-js/scalajs-compiler_2.13.12/1.17.0/scalajs-compiler_2.13.12-1.17.0.jar
[242] [warn] one warning found
[242] [info] done compiling
1 tasks failed
jsSite.test.fastLinkJSTest There were linking errors
jam01 commented 6 days ago

Can you share your project configuration or any further error stack trace?

Quafadas commented 6 days ago

The "project" is here;

https://github.com/Quafadas/json_schema_validation_test

here's more from the link error;

Referring to non-existent method java.nio.StringCharBuffer.chars()java.util.stream.IntStream
  dispatched from java.lang.CharSequence.chars()java.util.stream.IntStream
  called from io.github.jam01.json_schema.vocab.FormatAssertion.visitString(java.lang.CharSequence,int)scala.collection.immutable.Seq
  called from io.github.jam01.json_schema.vocab.FormatAssertion.visitString(java.lang.CharSequence,int)java.lang.Object
  dispatched from upickle.core.Visitor.visitString(java.lang.CharSequence,int)java.lang.Object
  called from ujson.Value$.transform(ujson.Value,upickle.core.Visitor)java.lang.Object
  called from ujson.Value.transform(upickle.core.Visitor)java.lang.Object
  called from ujson.Str.transform(upickle.core.Visitor)java.lang.Object
  dispatched from ujson.Readable.transform(upickle.core.Visitor)java.lang.Object
  called from ujson.Readable$.transform(ujson.Readable,upickle.core.Visitor)java.lang.Object
  called from ujson.Readable$.transform(java.lang.Object,upickle.core.Visitor)java.lang.Object
  dispatched from upickle.core.Transformer.transform(java.lang.Object,upickle.core.Visitor)java.lang.Object
  called from io.github.jam01.json_schema.package$.from(upickle.core.Transformer,java.lang.Object,io.github.jam01.json_schema.Uri,io.github.jam01.json_schema.Registry)io.github.jam01.json_schema.Schema
  called from validation$package$.validate()void
  called from static validate.main([java.lang.String)void
  called from core module module initializers

The linking error above, comes from the JS branch.

Without wishing to appear disheartening, the main branch also fails (on the JVM)

Exception in thread "main" java.util.NoSuchElementException: Unavailable schema urn:uuid:5003e9f0-34f8-45d8-83f5-3b10cbcaca7e#/definitions/TopLevelSpec
        at io.github.jam01.json_schema.Context.getSchOrThrow(Context.scala:52)
        at io.github.jam01.json_schema.Context.getSchOrThrow$(Context.scala:12)

If there's anything else, I can try to provide it, this is all obviously just for context. I'm obviously not expecting mircales, and my use case was rather complex. I gave the library a go on a Hail Mary :-).

jam01 commented 6 days ago

Try this

  val registry = new MutableRegistry
  val sch: Schema = json_schema.from(
    ujson.Readable,
    ujson.Readable.fromString(schema.text()),
    registry=registry
  )
  val validator: Visitor[?, OutputUnit] = json_schema.validator(sch, registry=registry)
Quafadas commented 4 days ago
Compiled project (Scala 3.6.1, JVM (23))
Exception in thread "main" java.util.NoSuchElementException: Unavailable schema urn:uuid:9bec752f-4850-48fb-91ba-5b2f2d727f0e#/definitions/TopLevelSpec
        at io.github.jam01.json_schema.Context.getSchOrThrow(Context.scala:52)
        at io.github.jam01.json_schema.Context.getSchOrThrow$(Context.scala:12)
        at io.github.jam01.json_schema.DefaultContext.getSchOrThrow(Context.scala:161)
        at io.github.jam01.json_schema.vocab.Core._refVis$lzyINIT1$$anonfun$1(Core.scala:21)
        at scala.Option.map(Option.scala:242)
        at io.github.jam01.json_schema.vocab.Core._refVis$lzyINIT1(Core.scala:21)
        at io.github.jam01.json_schema.vocab.Core._refVis(Core.scala:20)
        at io.github.jam01.json_schema.vocab.Core.visitString(Core.scala:64)
        at io.github.jam01.json_schema.vocab.Core.visitString(Core.scala:62)
        at io.github.jam01.json_schema.FFastObjectSchemaValidator.$anonfun$7(SchemaValidator.scala:94)
        at io.github.jam01.json_schema.FFastObjectSchemaValidator.visitString(SchemaValidator.scala:75)
        at io.github.jam01.json_schema.FFastObjectSchemaValidator.visitString(SchemaValidator.scala:94)
        at upickle.core.Visitor$Delegate.visitString(Visitor.scala:162)
        at ujson.Value$.transform(Value.scala:163)
        at ujson.Value.transform(Value.scala:110)
        at ujson.Value.transform$(Value.scala:9)
        at ujson.Str.transform(Value.scala:206)
        at validation$package$.validate(validation.scala:22)
        at validate.main(validation.scala:11)
Quafadas commented 4 days ago

This was after your suggestion (on the JVM)

On JS I still get linking errors.

jam01 commented 4 days ago

You missed to add the populated registry to the validator

val validator: Visitor[?, OutputUnit] = json_schema.validator(sch, registry=registry)
Quafadas commented 4 days ago

urgh, apologies, now this

Exception in thread "main" java.lang.ClassCastException: class io.github.jam01.json_schema.Obj cannot be cast to class io.github.jam01.json_schema.Schema (io.github.jam01.json_schema.Obj and io.github.jam01.json_schema.Schema are in unnamed module of loader 'app')
        at io.github.jam01.json_schema.ObjSchema.schBy0(ObjSchema.scala:275)
        at io.github.jam01.json_schema.ObjSchema.schBy0$(ObjSchema.scala:12)
jam01 commented 4 days ago

OK this is a lib incompatibility issue, I think. Let me look into this...

jam01 commented 3 days ago

OK! It is not a library issue, but an incompatibility issue with the schema. The library is only compatible with JSON Schema v2020-12 which defines definitions to be under a $defs key whereas I think from v4 until v2020-12 they could've been under definitions.

After modifying the schema per

sed -i 's/#\/definitions\//#\/$defs\//g' v5.21.0.json

plus adding some validation settings such as

val validator: Visitor[?, OutputUnit] =
  json_schema.validator(sch, config = Config(format=OutputFormat.Detailed, ffast=false), registry = registry)
val result: OutputUnit = ujson.Str("foo").transform(validator)
val dres = OutputUnitW.transform(result, StringRenderer()).toString
println(dres)

prints out the following errors, mostly about nested applicators expecting a list of objects

{"valid":false,"keywordLocation":"","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelSpec/anyOf","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/0","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/0/$ref","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/0/$ref/type","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelUnitSpec/type","instanceLocation":"","error":"Expected List(object), got string"}]}]},{"valid":false,"keywordLocation":"/$ref/anyOf/1","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/1/$ref","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/1/$ref/type","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelFacetSpec/type","instanceLocation":"","error":"Expected List(object), got string"}]}]},{"valid":false,"keywordLocation":"/$ref/anyOf/2","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/2/$ref","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/2/$ref/type","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelLayerSpec/type","instanceLocation":"","error":"Expected List(object), got string"}]}]},{"valid":false,"keywordLocation":"/$ref/anyOf/3","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/3/$ref","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/3/$ref/anyOf","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelRepeatSpec/anyOf","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/3/$ref/anyOf/0","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/3/$ref/anyOf/0/type","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelRepeatSpec/anyOf/0/type","instanceLocation":"","error":"Expected List(object), got string"}]},{"valid":false,"keywordLocation":"/$ref/anyOf/3/$ref/anyOf/1","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/3/$ref/anyOf/1/type","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelRepeatSpec/anyOf/1/type","instanceLocation":"","error":"Expected List(object), got string"}]}]}]}]},{"valid":false,"keywordLocation":"/$ref/anyOf/4","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/4/$ref","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/4/$ref/type","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelConcatSpec/type","instanceLocation":"","error":"Expected List(object), got string"}]}]},{"valid":false,"keywordLocation":"/$ref/anyOf/5","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/5/$ref","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/5/$ref/type","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelVConcatSpec/type","instanceLocation":"","error":"Expected List(object), got string"}]}]},{"valid":false,"keywordLocation":"/$ref/anyOf/6","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/6/$ref","instanceLocation":"","details":[{"valid":false,"keywordLocation":"/$ref/anyOf/6/$ref/type","absoluteKeywordLocation":"urn:uuid:bf9ad6b2-75a3-4cd6-b459-da59ae1ae2eb#/$defs/TopLevelHConcatSpec/type","instanceLocation":"","error":"Expected List(object), got string"}]}]}]}]}]}

In general the library is fully functional per the v2020-12 spec, but could definitely use a bit better docs. Ideally some common use cases compatibility with older schemas too, though that might be a slippery slope of undocumented functionality.

Quafadas commented 3 days ago

Well, it definitely looks like you've gotten it working :-). The message here looks like the right one - my next step would indeed have been to test it against a valid spec.

But ... I'm ... still struggling it seems.

Compiled project (Scala 3.6.1, JVM (23))
Exception in thread "main" java.lang.IllegalArgumentException: invalid location /$defs/TopLevelSpec
        at io.github.jam01.json_schema.ObjSchema$.io$github$jam01$json_schema$ObjSchema$$$refError(ObjSchema.scala:287)
        at io.github.jam01.json_schema.ObjSchema$.io$github$jam01$json_schema$ObjSchema$$$getOrThrow(ObjSchema.scala:283)
        at io.github.jam01.json_schema.ObjSchema.schBy0$$anonfun$1(ObjSchema.scala:264)

After having done the sed command above... dunno what I'm doing wrong :-(

jam01 commented 3 days ago

Oh right, the sed command rewrites the fragment references, then I manually edited the definitions key to read $defs. sed didn't catch it because it doesn't start with #/. Hope that makes sense.