Can't resolve path to sub-schemas in sub-folders

[duccioa]

Hello, I am trying to validate a json file with nested references. Let's say a 'user' json file with its 'address' sub-schema. 'address' also has a sub-schema. The sub-schemas are all contained in a 'sub-schema/' folder. The files are organised as following:

root/schemas/user
root/schemas/sub-schemas/address
root/schemas/sub-schemas/city

and they are referenced user -> sub-schemas/address -> sub-schemas/city

Here's the reproducible code in R which gives the error:

library(jsonvalidate)
library(testthat)

# Prepare paths and folder structure ----
root = file.path(tempdir(), "jsonvalidate")
dir = "schemas"
subdir = "sub-schemas"

dir.create(file.path(root, dir, subdir), recursive = TRUE, showWarnings = FALSE)

user_file = "user"
user_schema_path = file.path(root, dir, user_file)

address_file = "address"
address_rel_path = file.path(subdir, address_file)

city_file = "city"
city_rel_path = file.path(subdir, city_file)

# Json and schemas ----
json_to_validate = '
{
  "address":{
    "city":"Firenze"
  }
}
'
user_schema_ref = sprintf('
{
  "$schema": "http://json-schema.org/draft-07/schema",
  "$id": "file://%s",
  "type":"object",
  "required":["address"],
  "properties":{
    "address":{
      "$ref": "%s"
    }
  }
}
', user_schema_path, address_rel_path)

address_schema = sprintf('
{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type":"object",
  "properties":{
    "city":{
      "$ref": "%s"
    }
  }
}
', city_rel_path)

city_schema = sprintf('
{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type":"string",
  "enum":["Firenze"]
}
')

cat(user_schema_ref)
#> 
#> {
#>   "$schema": "http://json-schema.org/draft-07/schema",
#>   "$id": "file:///var/folders/yq/82fd9hrj1zs7kx2hqbqlg9280000gp/T//Rtmp0N5QNb/jsonvalidate/schemas/user",
#>   "type":"object",
#>   "required":["address"],
#>   "properties":{
#>     "address":{
#>       "$ref": "sub-schemas/address"
#>     }
#>   }
#> }
cat(address_schema)
#> 
#> {
#>   "$schema": "http://json-schema.org/draft-07/schema",
#>   "type":"object",
#>   "properties":{
#>     "city":{
#>       "$ref": "sub-schemas/city"
#>     }
#>   }
#> }
cat(city_schema)
#> 
#> {
#>   "$schema": "http://json-schema.org/draft-07/schema",
#>   "type":"string",
#>   "enum":["Firenze"]
#> }

# Write files ----
write(user_schema_ref, user_schema_path)
write(address_schema, file.path(root, dir, address_rel_path))
write(city_schema, file.path(root, dir, city_rel_path))

list.files(root, recursive = TRUE)
#> [1] "schemas/sub-schemas/address" "schemas/sub-schemas/city"   
#> [3] "schemas/user"

# Validate ----
json_validate(entry1, user_schema_path, engine = "ajv",
              verbose = TRUE, greedy = TRUE, 
              error = FALSE)
#> Error in context_eval(join(src), private$context, serialize, await): Error: can't resolve reference sub-schemas/city from id sub-schemas/address

^{Created on 2022-08-29 with reprex v2.0.2}

I managed to make it work if I put the files all in the same folder:

root/schemas/user
root/schemas/address
root/schemas/city

And I reference as following:

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "$id": "file:///var/folders/yq/82fd9hrj1zs7kx2hqbqlg9280000gp/T//Rtmpguupsl/jsonvalidate/schemas/user",
  "type":"object",
  "required":["address"],
  "properties":{
    "address":{
      "$ref": "address"
    }
  }
}

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "type":"object",
  "properties":{
    "city":{
      "$ref": "city"
    }
  }
}

But, as soon as I move the sub-schemas in a sub-folder I can't make it work.

Thank you!

[duccioa]

I have posted a question regarding the same issue on Stackoverflow, for the moment the answer is that it is not supported by {jsonvalidate}

https://stackoverflow.com/questions/73472005/multiple-level-nesting-in-json-schema-validation-with-sub-folders

[richfitz]

yes, there's something up with our logic for supporting this. As the docs linked by the SO user indicate, ajv requires that the wrapper library resolves the path references as ajv does not have access to the filesystem, and this is a bit fiddly.

We will resolve this at some point, but I'm afraid it's not at the top of our list at the moment. If you wanted to tackle it yourself, the code to change is in https://github.com/ropensci/jsonvalidate/blob/master/R/read.R and a test case could easily be fashioned from your nice reprex, a PR would be great if you have time

[richfitz]

Hi @duccioa, @r-ash has now fixed this in #62, can you give it a try please?

[duccioa]

Hello, Thanks for the quick fix. I have created a reprex for my current situation. It works with the new patch, where it would fail with the pre-patch version.

library(jsonvalidate)
parent = c(
  '{',
  '  "type":"object",',
  '  "required":["child1"],',
  '  "properties":{',
  '    "child1":{',
  '      "$ref":"sub/child1.json"',
  '    }',
  '  }',
  '}'
)
child1 = c(
  '{',
  '  "type":"object",',
  '  "properties":{',
  '    "child2":{',
  '      "$ref":"sub/child2.json"',
  '    }',
  '  }',
  '}'
)
child2 = c(
  '{',
  '  "type":"string",',
  '  "enum":["test"]',
  '}'
)

dir = file.path(tempdir(), "jsonvalidate_test")
schemadir = file.path(dir, "schemas")
schemasubdir1 = file.path(schemadir, "sub")
schemasubdir2 = file.path(schemasubdir1, "sub")
dir.create(schemasubdir2, recursive = TRUE, showWarnings = FALSE)  
writeLines(parent, file.path(schemadir, "parent.json"))  
writeLines(child1, file.path(schemasubdir1, "child1.json"))  
writeLines(child2, file.path(schemasubdir2, "child2.json"))  
list.files(schemadir, recursive = TRUE)
#> [1] "parent.json"         "sub/child1.json"     "sub/sub/child2.json"

json_validate('{"child1":{"child2":"test"}}', file.path(schemadir, "parent.json"), engine = "ajv")
#> [1] TRUE
json_validate('{"child1":{"child2":""}}', file.path(schemadir, "parent.json"), engine = "ajv")
#> [1] FALSE

unlink(dir, recursive = TRUE)

^{Created on 2022-09-21 with reprex v2.0.2}