search

monarch-initiative / phenol

phenol: Phenotype ontology library
https://phenol.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
23 stars 4 forks source link

Cannot load Mondo file #372

Closed mabeckwith closed 1 year ago

mabeckwith commented 2 years ago

org.monarchinitiative.phenol.base.PhenolRuntimeException: Error loading JSON at org.monarchinitiative.phenol.io.OntologyLoader.loadGraphDocument(OntologyLoader.java:94) at org.monarchinitiative.phenol.io.OntologyLoader.loadGraphDocument(OntologyLoader.java:75) at org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:43) at org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:35) at org.monarchinitiative.phenol.cli.demo.MondoDemo.run(MondoDemo.java:63) at org.monarchinitiative.phenol.cli.cmd.MondoDemoCommand.call(MondoDemoCommand.java:22) at org.monarchinitiative.phenol.cli.cmd.MondoDemoCommand.call(MondoDemoCommand.java:8) at picocli.CommandLine.executeUserObject(CommandLine.java:1933) at picocli.CommandLine.access$1200(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2332) at picocli.CommandLine$RunLast.handle(CommandLine.java:2326) at picocli.CommandLine$RunLast.handle(CommandLine.java:2291) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2159) at picocli.CommandLine.execute(CommandLine.java:2058) at org.monarchinitiative.phenol.cli.Main.main(Main.java:30) Caused by: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom, problem: java.lang.NullPointerException at [Source: (BufferedInputStream); line: 1847541, column: 5] (through reference chain: org.geneontology.obographs.core.model.GraphDocument$Builder["graphs"]->java.util.ArrayList[0]->org.geneontology.obographs.core.model.Graph$Json["logicalDefinitionAxioms"]->java.util.ArrayList[296]) at com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47) at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1907) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:587) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:610) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:647) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createUsingDelegate(StdValueInstantiator.java:306) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1382) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:347) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129) at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:225) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:197) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1383) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:347) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28) at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158) at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293) at com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3585) at org.monarchinitiative.phenol.io.obographs.OboGraphDocumentLoader.loadJson(OboGraphDocumentLoader.java:35) at org.monarchinitiative.phenol.io.OntologyLoader.loadGraphDocument(OntologyLoader.java:92) ... 14 more Caused by: java.lang.NullPointerException at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:878) at com.google.common.collect.ImmutableList$Builder.add(ImmutableList.java:787) at com.google.common.collect.ImmutableList$Builder.add(ImmutableList.java:749) at com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:441) at com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.java:834) at org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom$Builder.addAllRestrictions(LogicalDefinitionAxiom.java:462) at org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom.fromJson(LogicalDefinitionAxiom.java:260) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) at java.base/java.lang.reflect.Method.invoke(Method.java:577) at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call1(AnnotatedMethod.java:107) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:631) ... 39 more [ERROR] Command execution failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) at org.apache.commons.exec.DefaultExecutor.executeInternal (DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.execute (DefaultExecutor.java:166) at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:982) at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:929) at org.codehaus.mojo.exec.ExecMojo.execute (ExecMojo.java:457) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:104) at java.lang.reflect.Method.invoke (Method.java:577) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) at org.codehaus.classworlds.Launcher.main (Launcher.java:47)

pnrobinson commented 2 years ago

Cannot construct instance of org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom, problem: java.lang.NullPointerException @cmungall @julesjacobsen @ielis @matentzn -- Any idea what this is related to? The error is thrown in OboGraphDocumentLoader

by the last line

import com.fasterxml.jackson.databind.ObjectMapper;
(...)
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
(...)
 return OBJECT_MAPPER.readValue(inputStream, GraphDocument.class);
matentzn commented 2 years ago

Which version of obographs are you using?

mondo is generated with

https://github.com/ontodev/robot/blob/master/robot-core/pom.xml#L207

pnrobinson commented 2 years ago 
<dependency>
       <groupId>org.geneontology.obographs</groupId>
       <artifactId>obographs-core</artifactId>
       <version>0.3.0</version>
    </dependency>
julesjacobsen commented 2 years ago

@matentzn I thought you were going to use the updated version? There was a lot of work that went into that to fulfill your requirements for ordering and reproducibility. If robot is updated to use the 0.3.0 version, it ought not produce fields with null values.

matentzn commented 2 years ago

Yep, see here: https://github.com/ontodev/robot/pull/934

matentzn commented 2 years ago

@julesjacobsen does obographs have a CLI that can convert from OWL to obographs-json? If you provide me such a CLI with instructions to use, I will add obographs 0.3.0 conformant exports to all monarch ontologies.

julesjacobsen commented 2 years ago

@matentzn no it doesn't, but that does sound like a bit of a glaring oversight! I'll need you to do some testing for me...

https://github.com/geneontology/obographs/issues/80

matentzn commented 2 years ago

Great, will do. For testing efficiency, just provide me a jar somewhere and instructions to use and we will get it to work.

julesjacobsen commented 2 years ago

Hopefully this should be self-documenting: https://collect.qmul.ac.uk/down?t=4L7230DRLHH3DUGG/6P4DPGD2AVDBGU4NHFH8OA8

matentzn commented 2 years ago

Its great, thanks. Completely self documented :). It works fine. I personally prefer -o to be a filename rather than directory because of my data pipelines, but since we need this primarily right now (and for future iterations of the format), I am fine with the way you designed it.

Assuming you need these:

What is the best path forward?

  1. @mabeckwith and other could simply mirror the OWL file http://purl.obolibrary.org/obo/mondo.owl with wget and cycle it through obographs (obographs convert mondo.owl -f json) prior to injection into phenol.
  2. If I update the ontology pipelines with obographs, it will have to go into ODK, which will take more than 6 weeks to accomplish (when the next release is due).
  3. Alternative 3 is to produce an ad-hoc version of %.owl (mondo-og.json, hp-og.json etc) and attach them to the release as a temporary fix.

The best path forward is 1, until we have the new OGraphs in ROBOT I think, but let me know what you think!

pnrobinson commented 2 years ago

Thanks @matentzn -- so basically with #1 we will avoid this error? What was the cause? I think #1 should be fine and we can let the new version percolate through #2.

julesjacobsen commented 2 years ago

@matentzn what would you prefer instead of this?

I agree that option 1 followed naturally by option 2 seems the easiest course of action.

mabeckwith commented 2 years ago

Thanks. I've tried 2 things:

1) Mirroring the OWL file from http://purl.obolibrary.org/obo/mondo.owl, and converting to JSON. The conversion to JSON failed.

I couldn't install wget, so mirrored the OWL file from http://purl.obolibrary.org/obo/mondo.owl using curl instead. When I tried to convert that OWL file using obographs convert mondo.owl -f json, I got many errors:

Could not parse ontology.  Either a suitable parser could not be found, or parsing failed.  See parser logs below for explanation.
The following parsers were tried:
1) org.semanticweb.owlapi.rdf.rdfxml.parser.RDFXMLParser@c1fca1e
2) org.semanticweb.owlapi.owlxml.parser.OWLXMLParser@4c7a078
3) org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxOWLParser@5f574cc2
4) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.RioTurtleDocumentFormatFactory@95fd655c
5) org.semanticweb.owlapi.manchestersyntax.parser.ManchesterOWLSyntaxOntologyParser@7f34a967
6) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.NQuadsDocumentFormatFactory@6f9c39ad
7) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.RDFJsonDocumentFormatFactory@cd748dc3
8) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.TrigDocumentFormatFactory@27e81c
9) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.NTriplesDocumentFormatFactory@937ecd36
10) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.RDFJsonLDDocumentFormatFactory@dcacc47d
11) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.BinaryRDFDocumentFormatFactory@3bf24493
12) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.RioRDFXMLDocumentFormatFactory@69b9a3bc
13) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.N3DocumentFormatFactory@9a5
14) org.semanticweb.owlapi.rdf.turtle.parser.TurtleOntologyParser@7a356a0d
15) org.semanticweb.owlapi.rio.RioTrixParserFactory$TrixParserImpl : org.semanticweb.owlapi.formats.TrixDocumentFormatFactory@27e82d
16) org.semanticweb.owlapi.oboformat.OBOFormatOWLAPIParser@2fb5fe30
17) org.semanticweb.owlapi.krss2.parser.KRSS2OWLParser@7a94b64e
18) org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.RDFaDocumentFormatFactory@264e8d

Detailed logs:
--------------------------------------------------------------------------------
Parser: org.semanticweb.owlapi.rdf.rdfxml.parser.RDFXMLParser@c1fca1e
    Stack trace:
org.xml.sax.SAXParseException; systemId: file:/Users/beckwm/mondo.owl; lineNumber: 1; columnNumber: 50; White spaces are required between publicId and systemId.        org.semanticweb.owlapi.rdf.rdfxml.parser.RDFXMLParser.parse(RDFXMLParser.java:78)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:221)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1294)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1234)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntologyFromOntologyDocument(OWLOntologyManagerImpl.java:1185)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntologyFromOntologyDocument(OWLOntologyManagerImpl.java:1197)
        org.geneontology.obographs.cli.commands.Convert.call(Convert.java:60)
        org.geneontology.obographs.cli.commands.Convert.call(Convert.java:19)
        picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        picocli.CommandLine.access$1300(CommandLine.java:145)
White spaces are required between publicId and systemId.        java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204)
        java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:178)
        java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
        java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
        java.xml/com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1465)
        java.xml/com.sun.org.apache.xerces.internal.impl.XMLScanner.scanExternalID(XMLScanner.java:1065)
        java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.scanDoctypeDecl(XMLDocumentScannerImpl.java:645)
        java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:908)
        java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605)
        java.xml/com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
(...)
Parser: org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxOWLParser@5f574cc2
    Stack trace:
Encountered " <FULLIRI> "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\"> "" at line 1, column 1.
Was expecting one of:
    "Ontology" ...
    "Prefix" ...
     (Line 0)        org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxOWLParser.parse(OWLFunctionalSyntaxOWLParser.java:103)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:221)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1294)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1234)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntologyFromOntologyDocument(OWLOntologyManagerImpl.java:1185)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntologyFromOntologyDocument(OWLOntologyManagerImpl.java:1197)
        org.geneontology.obographs.cli.commands.Convert.call(Convert.java:60)
        org.geneontology.obographs.cli.commands.Convert.call(Convert.java:19)
        picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        picocli.CommandLine.access$1300(CommandLine.java:145)
(...)
Parser: org.semanticweb.owlapi.rio.RioParserImpl : org.semanticweb.owlapi.formats.RioTurtleDocumentFormatFactory@95fd655c
    Stack trace:
org.openrdf.rio.UnsupportedRDFormatException: No parser factory available for RDF format Turtle (mimeTypes=text/turtle, application/x-turtle; ext=ttl)        org.semanticweb.owlapi.rio.RioParserImpl.parse(RioParserImpl.java:194)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:221)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1294)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1234)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntologyFromOntologyDocument(OWLOntologyManagerImpl.java:1185)
        uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntologyFromOntologyDocument(OWLOntologyManagerImpl.java:1197)
        org.geneontology.obographs.cli.commands.Convert.call(Convert.java:60)
        org.geneontology.obographs.cli.commands.Convert.call(Convert.java:19)
        picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        picocli.CommandLine.access$1300(CommandLine.java:145)

(...etc for other parsers)

2) Converting a local mondo.owl that I had previously downloaded. The conversion was successful, but the converted JSON file couldn't be read.

I also tried converting a mondo.owl file that I had previously downloaded, and that file was able to be converted, but the converted JSON file has a null value somewhere:

Caused by: java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()" because "xrefs" is null
    at org.monarchinitiative.phenol.io.obographs.OboGraphTermFactory.mapXref(OboGraphTermFactory.java:150)
    at org.monarchinitiative.phenol.io.obographs.OboGraphTermFactory.convertToSynonyms(OboGraphTermFactory.java:133)
    at org.monarchinitiative.phenol.io.obographs.OboGraphTermFactory.constructTerm(OboGraphTermFactory.java:63)
    at org.monarchinitiative.phenol.io.obographs.OboGraphDocumentAdaptor$Builder.convertNodesToTerms(OboGraphDocumentAdaptor.java:160)
    at org.monarchinitiative.phenol.io.obographs.OboGraphDocumentAdaptor$Builder.build(OboGraphDocumentAdaptor.java:99)
    at org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:66)
    at org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:44)
    at org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:35)
    at org.monarchinitiative.l2ci.core.io.HPOParser.parse(HPOParser.java:26)
    at org.monarchinitiative.l2ci.core.io.HPOParser.<init>(HPOParser.java:17)
    at org.monarchinitiative.l2ci.gui.controller.MainController.loadFile(MainController.java:201)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
    ... 47 more
matentzn commented 2 years ago 
"<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\"> "" at l

This suggests you didn't use curl with follow redirect option. Better download manually or use curl with follow redirects option activated..

mabeckwith commented 2 years ago

Thanks. Using curl with the follow redirects option created a mondo.owl file that was able to be converted to a JSON, but trying to read that JSON file results in the same NullPointerException I saw with the other mondo.json file that was converted from a mondo.owl I had previously downloaded:

Caused by: java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()" because "xrefs" is null
    at org.monarchinitiative.phenol.io.obographs.OboGraphTermFactory.mapXref(OboGraphTermFactory.java:150)
    at org.monarchinitiative.phenol.io.obographs.OboGraphTermFactory.convertToSynonyms(OboGraphTermFactory.java:133)
    at org.monarchinitiative.phenol.io.obographs.OboGraphTermFactory.constructTerm(OboGraphTermFactory.java:63)
    at org.monarchinitiative.phenol.io.obographs.OboGraphDocumentAdaptor$Builder.convertNodesToTerms(OboGraphDocumentAdaptor.java:160)
    at org.monarchinitiative.phenol.io.obographs.OboGraphDocumentAdaptor$Builder.build(OboGraphDocumentAdaptor.java:99)
    at org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:66)
    at org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:44)
    at org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:35)
    at org.monarchinitiative.l2ci.core.io.HPOParser.parse(HPOParser.java:26)
    at org.monarchinitiative.l2ci.core.io.HPOParser.<init>(HPOParser.java:17)
    at org.monarchinitiative.l2ci.gui.controller.MainController.loadFile(MainController.java:201)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
    ... 47 more

Also, like with the other file, when the JSON is being read I get many warnings like this before the NullPointerException

2022-04-20 13:06:33.743  WARN 64443 --- [lication Thread] o.m.p.i.o.OboGraphDocumentAdaptor        : No matching curie found for id: http://identifiers.org/hgnc/10001
2022-04-20 13:06:33.744  WARN 64443 --- [lication Thread] o.m.p.i.o.OboGraphDocumentAdaptor        : No matching curie found for id: http://identifiers.org/hgnc/10004
2022-04-20 13:06:33.744  WARN 64443 --- [lication Thread] o.m.p.i.o.OboGraphDocumentAdaptor        : No matching curie found for id: http://identifiers.org/hgnc/10006
2022-04-20 13:06:33.744  WARN 64443 --- [lication Thread] o.m.p.i.o.OboGraphDocumentAdaptor        : No matching curie found for id: http://identifiers.org/hgnc/10012
2022-04-20 13:06:33.744  WARN 64443 --- [lication Thread] o.m.p.i.o.OboGraphDocumentAdaptor        : No matching curie found for id: http://identifiers.org/hgnc/10013
2022-04-20 13:06:33.744  WARN 64443 --- [lication Thread] o.m.p.i.o.OboGraphDocumentAdaptor        : No matching curie found for id: http://identifiers.org/hgnc/10023
(...etc)
matentzn commented 2 years ago

Back to @julesjacobsen .. I have never parsed an OBOGraphs document, but we have done hundreds of roundtripping tests on his code, so a bit surprising..

julesjacobsen commented 2 years ago

This isn't an obographs problem anymore. The error starts with trying to parse MONDO using an HPO parser org.monarchinitiative.l2ci.core.io.HPOParser.parse(HPOParser.java:26). This might have some preconceived ideas about what the ontology should look like or there is a bug at org.monarchinitiative.phenol.io.obographs.OboGraphTermFactory.mapXref(OboGraphTermFactory.java:150) or its a problem caused by the HGNC curies not being recognised.

What is the l2ci package?

julesjacobsen commented 2 years ago

This has been resolved with @mabeckwith - there was an inconsistency with the OboGraphTermFactory.mapXref error which was fixed and the CurieUtil has been updated to allow for any CURIE mappings to be updated/ overridden by the user.

ielis commented 1 year ago

Hi @matentzn @julesjacobsen , so sorry to bother you but I am not sure if I am doing something stupid or if there are still some outstanding issues with obographs vs. Mondo.

The parsing does not work from some reason on my end. When running the following on the most recent develop:

// mondo.json is v2022-12-01
Path path = Path.of("mondo.json");
Ontology mondo = OntologyLoader.loadOntology(path.toFile());

I receive the following error:

org.monarchinitiative.phenol.base.PhenolRuntimeException: Error loading JSON

    at org.monarchinitiative.phenol.io/org.monarchinitiative.phenol.io.OntologyLoader.loadGraphDocument(OntologyLoader.java:94)
    at org.monarchinitiative.phenol.io/org.monarchinitiative.phenol.io.OntologyLoader.loadGraphDocument(OntologyLoader.java:75)
    at org.monarchinitiative.phenol.io/org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:43)
    at org.monarchinitiative.phenol.io/org.monarchinitiative.phenol.io.OntologyLoader.loadOntology(OntologyLoader.java:35)
    at org.monarchinitiative.phenol.io/org.monarchinitiative.phenol.io.OntologyLoaderMondoTest.loadFull(OntologyLoaderMondoTest.java:66)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
    at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
    at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
    at 

// A bunch of JUnit-related exceptions 

Caused by: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom`, problem: `java.lang.NullPointerException`
 at [Source: (BufferedInputStream); line: 1829837, column: 5] (through reference chain: org.geneontology.obographs.core.model.GraphDocument$Builder["graphs"]->java.util.ArrayList[0]->org.geneontology.obographs.core.model.Graph$Json["logicalDefinitionAxioms"]->java.util.ArrayList[308])
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1907)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:587)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:610)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:647)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createUsingDelegate(StdValueInstantiator.java:306)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1382)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:347)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:225)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:197)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1383)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:347)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:244)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:28)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeSetAndReturn(MethodProperty.java:158)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.vanillaDeserialize(BuilderBasedDeserializer.java:293)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.BuilderBasedDeserializer.deserialize(BuilderBasedDeserializer.java:217)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3585)
    at org.monarchinitiative.phenol.io/org.monarchinitiative.phenol.io.obographs.OboGraphDocumentLoader.loadJson(OboGraphDocumentLoader.java:35)
    at org.monarchinitiative.phenol.io/org.monarchinitiative.phenol.io.OntologyLoader.loadGraphDocument(OntologyLoader.java:92)
    ... 71 more
Caused by: java.lang.NullPointerException
    at com.google.common@30.1-jre/com.google.common.base.Preconditions.checkNotNull(Preconditions.java:878)
    at com.google.common@30.1-jre/com.google.common.collect.ImmutableList$Builder.add(ImmutableList.java:787)
    at com.google.common@30.1-jre/com.google.common.collect.ImmutableList$Builder.add(ImmutableList.java:749)
    at com.google.common@30.1-jre/com.google.common.collect.ImmutableCollection$Builder.addAll(ImmutableCollection.java:441)
    at com.google.common@30.1-jre/com.google.common.collect.ImmutableList$Builder.addAll(ImmutableList.java:834)
    at obographs.core@0.3.0/org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom$Builder.addAllRestrictions(LogicalDefinitionAxiom.java:462)
    at obographs.core@0.3.0/org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom.fromJson(LogicalDefinitionAxiom.java:260)
    at jdk.internal.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call1(AnnotatedMethod.java:107)
    at com.fasterxml.jackson.databind@2.12.1/com.fasterxml.jackson.databind.deser.std.StdValueInstantiator._createUsingDelegate(StdValueInstantiator.java:631)
    ... 96 more

The stack trace seems to be pointing at a null value on line #1829837.

When using sed to view the neighborhood (+- 10 lines) of the offending line:

sed -n '1829827,1829847p;1829847q' mondo.json

I see the following:

      "definedClassId" : "http://purl.obolibrary.org/obo/MONDO_0000675",
      "genusIds" : [ "http://purl.obolibrary.org/obo/MONDO_0005638" ],
      "restrictions" : [ {
        "propertyId" : "http://purl.obolibrary.org/obo/RO_0004021",
        "fillerId" : "http://purl.obolibrary.org/obo/GO_0019233"
      } ]
    }, {
      "definedClassId" : "http://purl.obolibrary.org/obo/ENVO_09000016",
      "genusIds" : [ "http://purl.obolibrary.org/obo/PATO_0000033" ],
      "restrictions" : [ null ]
    }, {
      "definedClassId" : "http://purl.obolibrary.org/obo/UBERON_0006937",
      "genusIds" : [ "http://purl.obolibrary.org/obo/UBERON_0000483" ],
      "restrictions" : [ {
        "propertyId" : "http://purl.obolibrary.org/obo/BFO_0000050",
        "fillerId" : "http://purl.obolibrary.org/obo/UBERON_0001846"
      } ]
    }, {
      "definedClassId" : "http://purl.obolibrary.org/obo/MONDO_0000931",
      "genusIds" : [ "http://purl.obolibrary.org/obo/MONDO_0000001" ],
      "restrictions" : [ {

Do you have any suggestions how to fix this? Thanks a lot & cheers..

julesjacobsen commented 1 year ago

@ielis This is odd.

wget http://purl.obolibrary.org/obo/mondo.owl
java -jar obographs-cli-0.3.0.jar convert mondo.owl mondo.json
java -jar obographs-cli-0.3.0.jar validate mondo.json
mondo.json - OK
grep 'null'
          "val" : "null cell"
          "val" : "null cell",
... and a bunch more valid null strings

Superficially, this looks OK. However...

wget http://purl.obolibrary.org/obo/mondo.json
java -jar obographs-cli-0.3.0.jar validate mondo.json
mondo.json - ERROR: Cannot construct instance of `org.geneontology.obographs.core.model.axiom.LogicalDefinitionAxiom`, problem: `java.lang.NullPointerException`
 at [Source: (File); line: 1835151, column: 5] (through reference chain: org.geneontology.obographs.core.model.GraphDocument$Builder["graphs"]->java.util.ArrayList[0]->org.geneontology.obographs.core.model.Graph$Json["logicalDefinitionAxioms"]->java.util.ArrayList[309])
grep 'null'
... a bunch of valid nulls and then
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
      "restrictions" : [ null ]
... and quite a few more

So my question for @matentzn is what is going on with the transformation of mondo.owl to mondo.json in robot?

matentzn commented 1 year ago

This is the POM:

https://github.com/ontodev/robot/blob/master/robot-core/pom.xml#L211

Here is the code:

https://github.com/ontodev/robot/blob/master/robot-core/src/main/java/org/obolibrary/robot/IOHelper.java#L1624

Anything glaringly wrong?

julesjacobsen commented 1 year ago

@ielis @matentzn @pnrobinson @mabeckwith I'm moving this into the obobraphs issue tracker issue #96 to consolidate the issue threads.

matentzn commented 1 year ago

This was a stupid mistake caused by my ignorance: https://github.com/monarch-initiative/mondo/pull/6229/files

Will be solved shortly.

matentzn commented 1 year ago

Here is a test file generated by the new pipeline (based on the current mondo release): https://www.dropbox.com/s/iwnthsmnz63wnhu/mondo.json?dl=0

At the next Mondo release the json will work correctly Sorry about that!