handling $ref pointers to other schemas?

[computableinsights]

Hi James and Sunlight Labs,

Is there support for $ref pointers in validictory? This is defined in draft-03 of the json-schema spec. It allows a schema to define types by referring to a schema at a URL.

Here is an example of such a schema:

http://trec-kba.org/schemas/v1.0/stream-item.json

which has a $ref to other schemas, like this one:

http://trec-kba.org/schemas/v1.0/content-item.json

jrf

[jamesturk]

we don't, it adds a decent amount of complexity and hasn't thus far been requested (and there are lots of dumb things in jsonschema so I've taken the position of only implementing them when people have a use case)

implementation shouldn't be that hard, i guess it'd best be handled by a preprocessing step that looked for $ref keys and came back with a completed dict. if you're looking to use it - i'd accept a patch or I can work to implement it myself but i'd have to warn that might take a while as it isn't a huge priority.

[computableinsights]

I can look into writing a patch after I understand a bit more about how all these different validators work.

I'm intrigued by your comment about dumb things in jsonschemas. Since it is stalled on draft-03, perhaps it is time for a trimming of bloated parts. Do you have a list of dumb things? Or a heuristic for detecting which parts are dumb? Is it as simple, as "nobody uses that" or is it more "nobody should use that part"?

[jamesturk]

$ref would need to work a bit differently than what is already there, when $ref is found it'd need to basically be replaced with a dict from a urlrequest

i think it'd make the most sense to preprocess the entire schema, replacing $refs with the fetched data

[jamesturk]

and regarding the bloated parts, none of them that i've implemented are that bad, but $ref and $id, and some of the other non-validation things in the spec are the ones that peresonally i've never seen a use for and up until now i haven't had any requests for

[vitaly-krugl]

The use case for $ref is quite important in my experience. It's very error-prone/unmaintainable to maintain copies of the same subschemaS inside higher-level schemas. It has the same problems as copying/pasting the same code into different functions/methods throughout a package versus abstracting and sharing.

[jamesturk]

This is easily done in Python by using references to dictionaries instead of formal URL references, it has worked well for a lot of people including projects I've worked on.

[vitaly-krugl]

Hi @jamesturk -- Our schemas are shared between programs written in different computer languages and live in files as json text. How can I use references to dictionaries in this context?

Thank you, Vitaly

[jamesturk]

You can load the dictionaries from json and then inject them as needed. If you want to try to write up a patch that does this automatically for $ref the best I can say is that I'd consider it if it was sure to not introduce security vulnerabilities.

-James

On Mon, Jun 24, 2013 at 5:49 PM, vitaly-krugl notifications@github.comwrote:

Hi @jamesturk https://github.com/jamesturk -- Our schemas are shared between programs written in different computer languages and live in files as json text. How can I use references to dictionaries in this context?

Thank you, Vitaly

— Reply to this email directly or view it on GitHubhttps://github.com/sunlightlabs/validictory/issues/20#issuecomment-19938539 .

[dmr]

Sorry for commenting on a closed issue but I found this issue via google and decided to push my solution to "the $ref problem" to github in case anybody else needs a solution to this: https://github.com/dmr/validictory_preprocess_ref

[iandanforth]

For anyone else who get's here you may want to look at https://github.com/Julian/jsonschema as a replacement for validictory that supports up to draft04

[vitaly-krugl]

Thanks for mentioning it @iandanforth!

[dmr]

I use validictory in production with $ref without problems and it's working really good. Are you missing any other draft4 features besides $ref @iandanforth?

[iandanforth]

@dmr Just that, and your solution almost works for my use-case. I also need to reference other files, not just internal refs. the RefResolver in jsonschema handles this, but I'd much prefer a pre-processor like yours.

[iandanforth]

@dmr nevermind I just figured out how to use https://github.com/gazpachoking/jsonref to do this.

import jsonref
import validictory

# Load that schema containing $refs
with open(<path_to_schema_file>) as fh:
    base_uri = "file://" + <path_to_base_schema_file> + "/"
    schema = jsonref.load(fh, base_uri)

# Validate the data against the schema
if not validictory.validate(data, schema):
    print("Data is valid")