search

geneontology / obographs

Basic and Advanced OBO Graphs: specification and reference implementation
64 stars 12 forks source link

NullPointerException in `org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom` #96

Closed matentzn closed 1 year ago

matentzn commented 1 year ago

(from @pnrobinson , on slack)

mondo.json appears to have a bug

When trying to load mondo.json (https://github.com/monarch-initiative/mondo/releases/download/v2023-04-04/mondo.json), created with obographs version 0.3.0, we get this:

/Users/robinp/Downloads/l4ci/data/mondo.json
org.monarchinitiative.phenol.base.PhenolRuntimeException: Error loading JSON
        at [org.monarchinitiative.phenol.io](http://org.monarchinitiative.phenol.io/).OntologyLoader.loadGraphDocument(OntologyLoader.java:109)
        at [org.monarchinitiative.phenol.io](http://org.monarchinitiative.phenol.io/).OntologyLoader.loadGraphDocument(OntologyLoader.java:90)
        at [org.monarchinitiative.phenol.io](http://org.monarchinitiative.phenol.io/).OntologyLoader.loadOntology(OntologyLoader.java:43)
        at [org.monarchinitiative.l4ci.core.io](http://org.monarchinitiative.l4ci.core.io/).HPOParser.parse(HPOParser.java:32)
        at [org.monarchinitiative.l4ci.core.io](http://org.monarchinitiative.l4ci.core.io/).HPOParser.<init>(HPOParser.java:20)
        at org.monarchinitiative.l4ci.cli.cmd.BenchmarkCommand.loadOntologies(BenchmarkCommand.java:298)
        at org.monarchinitiative.l4ci.cli.cmd.GeneAnalysisCommand.call(GeneAnalysisCommand.java:80)
        at org.monarchinitiative.l4ci.cli.cmd.GeneAnalysisCommand.call(GeneAnalysisCommand.java:40)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1933)
        at picocli.CommandLine.access$1200(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2326)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2291)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159)
        at picocli.CommandLine.execute(CommandLine.java:2058)
        at org.monarchinitiative.l4ci.cli.Main.main(Main.java:29)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom`, problem: `java.lang.NullPointerException`
 at [Source: (BufferedInputStream); line: 1835151, column: 5] (through reference chain: org.geneontology.obographs.core.model.GraphDocument$Builder["graphs"]->java.util.ArrayList[0]->org.geneontology.obographs.core.model.Graph$Json["logicalDefinitionAxioms"]->java.util.ArrayList[309])
        at com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47)
        at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:2047)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:587)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:610)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:647)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createUsingDelegate(StdValueInstantiator.java:306)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1397)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:351)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:184)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
        at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:313)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:214)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:186)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1398)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:351)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:184)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:355)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
        at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
        at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158)
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293)
        at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3666)
        at [org.monarchinitiative.phenol.io](http://org.monarchinitiative.phenol.io/).obographs.OboGraphDocumentLoader.loadJson(OboGraphDocumentLoader.java:35)
        at [org.monarchinitiative.phenol.io](http://org.monarchinitiative.phenol.io/).OntologyLoader.loadGraphDocument(OntologyLoader.java:107)
        ... 21 more
Caused by: java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:782)
        at com.google.common.collect.ImmutableCollection$ArrayBasedBuilder.add(ImmutableCollection.java:479)
        at com.google.common.collect.ImmutableList$Builder.add(ImmutableList.java:743)
        at com.google.common.collect.ImmutableList$Builder.add(ImmutableList.java:719)
        at com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:419)
        at com.google.common.collect.ImmutableCollection$ArrayBasedBuilder.addAll(ImmutableCollection.java:502)
        at com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.java:758)
        at org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom$Builder.addAllRestrictions(LogicalDefinitionAxiom.java:462)
        at org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom.fromJson(LogicalDefinitionAxiom.java:260)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call1(AnnotatedMethod.java:109)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:631)
        ... 46 more

I think the error is coming from obographs although the exception is thrown from phenol

julesjacobsen commented 1 year ago

This duplicates https://github.com/monarch-initiative/phenol/issues/372

See my comment there

wget http://purl.obolibrary.org/obo/mondo.owl
java -jar obographs-cli-0.3.0.jar convert mondo.owl mondo.json
java -jar obographs-cli-0.3.0.jar validate mondo.json
mondo.json - OK
grep 'null'
          "val" : "null cell"
          "val" : "null cell",
... and a bunch more valid null strings

Superficially, this looks OK. However...

wget http://purl.obolibrary.org/obo/mondo.json
java -jar obographs-cli-0.3.0.jar validate mondo.json
mondo.json - ERROR: Cannot construct instance of `org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom`, problem: `java.lang.NullPointerException`
 at [Source: (File); line: 1835151, column: 5] (through reference chain: org.geneontology.obographs.core.model.GraphDocument$Builder["graphs"]->java.util.ArrayList[0]->org.geneontology.obographs.core.model.Graph$Json["logicalDefinitionAxioms"]->java.util.ArrayList[309])
grep 'null'
... a bunch of valid nulls and then
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
... and quite a few more

So my question for @matentzn is what is going on with the transformation of mondo.owl to mondo.json in robot?

julesjacobsen commented 1 year ago

Moving this here:

https://github.com/monarch-initiative/phenol/issues/372#issuecomment-1527775440

This is the POM:

https://github.com/ontodev/robot/blob/master/robot-core/pom.xml#L211

Here is the code:

https://github.com/ontodev/robot/blob/master/robot-core/src/main/java/org/obolibrary/robot/IOHelper.java#L1624

Anything glaringly wrong?

julesjacobsen commented 1 year ago

@matentzn Is your pipeline using that version of robot? If not, use the latest version. If it is, then what is happening to the ontology in between owlapi loading it from owl and it being written out to json?

julesjacobsen commented 1 year ago 
docker pull docker.io/obolibrary/robot:latest
docker run -v $PWD/:/work -w /work --rm -ti obolibrary/robot robot --version
ROBOT version 1.9.1
docker run -v $PWD/:/work -w /work --rm -ti obolibrary/robot robot convert --input mondo.owl --output mondo-robot-1.9.1.json --format json
java -jar obographs-cli-0.3.0.jar validate mondo-robot-1.9.1.json 
mondo-robot-1.9.1.json - OK
diff mondo-obog.json mondo-robot-1.9.1.json
... no difference

So, it looks like using the latest version of ROBOT produces the exact same, valid, output as the obographs-cli. This suggests that the issue is sitting somewhere in the MONDO pipeline where there is an older version of ROBOT being used to convert the files.

matentzn commented 1 year ago

😱

https://github.com/monarch-initiative/mondo/blob/master/src/ontology/Makefile#L253

I am so sorry! I never ever knew that! This is insane.

100% my fault, sorry for sending you on a goose chase, and thanks for debugging... Feeling very bad.

matentzn commented 1 year ago

This should take care of this:

https://github.com/monarch-initiative/mondo/pull/6229/files

THANK YOU @julesjacobsen

julesjacobsen commented 1 year ago

Well, so long as owltools isn't needed for some other reason?

Can you also please confirm that the output .JSON file passes the obographs-cli validate command. I can run this for you if you want, alternatively I'll generate a checksum of the latest one direct from obographs-cli/ robot.

matentzn commented 1 year ago

Here is the generated file: https://www.dropbox.com/s/iwnthsmnz63wnhu/mondo.json?dl=0

julesjacobsen commented 1 year ago

Close enough...

diff mondo-obog.json mondo-niko.json 
747807c747807
<           "val" : "https://github.com/monarch-initiative/mondo/issues/4521"
---
>           "val" : "https://github.com/monarch-initiative/mondo/issues/5373"
747810c747810
<           "val" : "https://github.com/monarch-initiative/mondo/issues/5373"
---
>           "val" : "https://github.com/monarch-initiative/mondo/issues/4521"
matentzn commented 1 year ago

Does this mean that there is an ordering issue? Interesting. Maybe divergent Java versions?

julesjacobsen commented 1 year ago

It's unlikely to be a Java version thing. More likely we used a slightly different version of the ontology or there is an owl ordering issue, although obographs sorts the input, so this should be consistent.