search

jhthorsen / json-validator

:cop: Validate data against a JSON schema
https://metacpan.org/release/JSON-Validator
57 stars 59 forks source link

oneOf does not work properly in version 0.96 #59

Closed ckongEbi closed 7 years ago

ckongEbi commented 7 years ago

Hi, by upgrading from version 0.92=>0.97, we hit into the error below, which wasn't happening before:

Validation error on evidence line 639 : $VAR1 = bless( { 'path' => '/', 'message' => 'allOf failed: allOf failed: All of the oneOf rules match.' }, 'JSON::Validator::Error' );

We think it might be a new bug, this is the schema we use

https://github.com/opentargets/json_schema/blob/master/src/evidence/base.json#L24

and a single line of our json data, thanks.

{"disease": {"name": "breast adenocarcinoma", "source_name": "breast adenocarcinoma", "id": "http://www.ebi.ac.uk/efo/EFO_0000304"}, "sourceID": "eva_somatic", "target": {"activity": "http://identifiers.org/cttv.activity/damaging_to_target", "id": "http://identifiers.org/ensembl/ENSG00000023287", "target_type": "http://identifiers.org/cttv.target/gene_variant"}, "literature": {"references": [{"lit_id": "http://europepmc.org/abstract/MED/12068296"}]}, "evidence": {"urls": [{"nice_name": "Further details in ClinVar database", "url": "http://www.ncbi.nlm.nih.gov/clinvar/RCV000004183"}], "date_asserted": "2016-10-03T00:00:00", "known_mutations": [{"preferred_name": "splice_acceptor_variant", "functional_consequence": "http://purl.obolibrary.org/obo/SO_0001574"}], "evidence_codes": ["http://purl.obolibrary.org/obo/ECO_0000205"], _"resourcescore": {"type": "probability", "value": 1}, "is_associated": true, "provenance_type": {"database": {"version": "1.0", "dbxref": {"version": "2015-04", "id": "http://identifiers.org/clinvar", "url": "http://identifiers.org/clinvar.record/RCV000004183"}, "id": "EVA"}, "expert": {"statement": "Primary submitter of data", "status": true}, "literature": {"references": [{"lit_id": "http://europepmc.org/abstract/MED/12068296"}]}}}, "unique_association_fields": {"phenotype": "http://www.ebi.ac.uk/efo/EFO_0000304", "clinvarAccession": "RCV000004183", "gene": "ENSG00000023287", "alleleOrigin": "somatic"}, "validated_against_schema_version": "1.2.5", "type": "somatic_mutation", "access_level": "public"}

jhthorsen commented 7 years ago

Thanks for reporting in.

Could you help me help you? If so, can you do this:

ckongEbi commented 7 years ago

@jhthorsen yes, but i think the archive link is not working, please have a look.

Fetching https://github.com/jhthorsen/json-validator/archive/0.93.tar.gz ... FAIL ! Download https://github.com/jhthorsen/json-validator/archive/0.93.tar.gz failed. Retrying ... ! Download https://github.com/jhthorsen/json-validator/archive/0.93.tar.gz failed. Retrying ... ! Download https://github.com/jhthorsen/json-validator/archive/0.93.tar.gz failed. Retrying ... ! Failed to download https://github.com/jhthorsen/json-validator/archive/0.93.tar.gz ! Failed to fetch distribution

Cheers

ckongEbi commented 7 years ago

@jhthorsen what i've located is that the issue appear to be introduced in 0.96. 0.95 runs fine without issue.

0.96 2017-03-06T20:53:35+0100

jhthorsen commented 7 years ago

What about a smaller test case?

ckongEbi commented 7 years ago

Example an extract of the JSON file,

"resource_score": {
  "value": 1,
  "type": "probability"
},

which corresponds to this schema, does this helps?

https://github.com/opentargets/json_schema/blob/master/src/evidence/association_score/probability.json

jhthorsen commented 7 years ago

Not really. I don't see any "oneOf" in that schema.

ckongEbi commented 7 years ago

Please see this https://github.com/opentargets/json_schema/blob/master/src/evidence/base.json which is parent of the former..

wbazant commented 7 years ago

@jhthorsen Here is a smaller case : out.txt as an output of this program (with a debug flag on):

my $validator = JSON::Validator->new;
$validator->schema($schemaUri);
print STDOUT Dumper($validator->{"schema"});

I had a go at debugging why this happens but the furthest I got is that in https://github.com/jhthorsen/json-validator/blob/master/lib/JSON/Validator.pm#L298 $look_in = $self->_load_schema($ref, $namespace); is fine, and then look_in gets overwritten with the other entry in the next line $look_in = $self->_resolve_schema($look_in, $look_in->data->{id} || $namespace);

jhthorsen commented 7 years ago

What do you mean with "$look_in is fine"? Is "fine" the same as a "true" value..? If that is the case, how can it go into the if (!$look_in) block?

wbazant commented 7 years ago

@jhthorsen sorry I haven't linked very precisely let me try improve the writeup of this issue. L299 $look_in = $self->_load_schema($ref, $namespace); is the last thing that I think executes correctly. $look_in is falsey after L297, goes into the conditions, is fine - by which I mean that when I look at the dump of that variable, it looks like the Perl version of the JSON object that represents the schema. After L300 executes, this is no longer the case - the dump of $look_in is mangled - all the array entries for where oneOf is become the same. Then after I let the snippet finish, the final output is wrong in the same way.

I squinted at the program for a while to see how it can happen but was thwarted by my lack of Perl skill.

Again, to reproduce:

my $validator = JSON::Validator->new;
$validator->schema("https://github.com/opentargets/json_schema/blob/master/src/evidence/base.json");
print STDOUT Dumper($validator->{"schema"});
wbazant commented 7 years ago

I think the cause of error is the same as #65 - I was super confused when writing the above but also observed the schema loaded up being incorrect. I propose to merge this issue with #65.

jhthorsen commented 7 years ago

I'm closing this after the commit from @khvzak. Please re-open if that didn't help. (Remember to test from "master" branch and not from CPAN)