search

Knowledge-Graph-Hub / kg-obo

A package to transform all OBO ontologies into KGX TSV format and OBO json, and put the transformed graph in KGhub
https://knowledge-graph-hub.github.io/kg-obo/getting_started.html
GNU General Public License v3.0
28 stars 2 forks source link

FOODON v2024-02-21 does not convert to obojson #212

Closed caufieldjh closed 6 months ago

caufieldjh commented 6 months ago

Describe the bug

In the last build, ROBOT's attempts to translate FOODON v2024-02-21 to obojson fail repeatedly.

Here's how it went:

20:21:49  foodon
20:21:49  http://purl.obolibrary.org/obo/foodon.owl
20:21:49  Current VersionIRI for foodon: https://github.com/FoodOntology/foodon/releases/download/v2024-02-21/foodon.owl
20:21:49  Current version for foodon: v2024-02-21
20:21:49  In foodon, used this value for version: versionIRI
20:21:49  Don't have this version of foodon yet - will transform.
20:21:49  No imports found for foodon.
20:21:49  Making directory data/foodon.
20:21:49  Making directory data/foodon/v2024-02-21.
20:21:49  Completed download from http://purl.obolibrary.org/obo/foodon.owl to /tmp/foodondx4znj8_.
20:21:49  Moving from /tmp/foodondx4znj8_ to data/foodon/v2024-02-21/foodon.owl.
20:21:49  ROBOT preprocessing: relax foodon
20:21:49  Relaxing /tmp/foodondx4znj8_ to /tmp/tmpkmsuo93a_foodon_relaxed.owl...
20:21:49  Complete.
20:21:49  Before relax: 725851 lines. After relax: 736008 lines.
20:21:49  ROBOT preprocessing: node ID normalization on foodon
20:21:49  Retrieving entity names in /tmp/tmpkmsuo93a_foodon_relaxed.owl...
20:21:49  Exported IDs to /tmp/tmpkmsuo93a_foodon_relaxed.owl.ids.csv.
20:21:49  Found 21 unexpected identifiers.
20:21:49  Will normalize 8 identifiers.
20:21:49  Wrote IRI maps to data/foodon/v2024-02-21/update_id_maps.tsv.
20:21:49  ROBOT preprocessing: convert foodon
20:21:49  Converting /tmp/tmpkmsuo93a_foodon_relaxed.owl to data/foodon/v2024-02-21/foodon.json...
20:21:49  ROBOT encountered an error: 
20:21:49  
20:21:49    RAN: /var/lib/jenkins/workspace/knowledge-graph-hub_kg-obo_main/gitrepo/robot convert --input /tmp/tmpkmsuo93a_foodon_relaxed.owl --format json --output data/foodon/v2024-02-21/foodon.json
20:21:49  
20:21:49    STDOUT:
20:21:49  OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
20:21:49  For details see: http://robot.obolibrary.org/errors#obo-graph-error
20:21:49  Use the -vvv option to show the stack trace.
20:21:49  Use the --help option to see usage information.
20:21:49  
20:21:49  
20:21:49    STDERR:
20:21:49  
20:21:49  Will try to repair...
20:21:49  ROBOT encountered another error: 
20:21:49  
20:21:49    RAN: /var/lib/jenkins/workspace/knowledge-graph-hub_kg-obo_main/gitrepo/robot remove --input /tmp/tmpkmsuo93a_foodon_relaxed.owl --select object-properties --output data/foodon/v2024-02-21/foodon.json
20:21:49  
20:21:49    STDOUT:
20:21:49  OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
20:21:49  For details see: http://robot.obolibrary.org/errors#obo-graph-error
20:21:49  Use the -vvv option to show the stack trace.
20:21:49  Use the --help option to see usage information.
20:21:49  
20:21:49  
20:21:49    STDERR:
20:21:49  
20:21:49  ROBOT encountered yet another error: 
20:21:49  
20:21:49    RAN: /var/lib/jenkins/workspace/knowledge-graph-hub_kg-obo_main/gitrepo/robot remove --input /tmp/tmpkmsuo93a_foodon_relaxed.owl --term rdfs:comment --output data/foodon/v2024-02-21/foodon.json
20:21:49  
20:21:49    STDOUT:
20:21:49  OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
20:21:49  For details see: http://robot.obolibrary.org/errors#obo-graph-error
20:21:49  Use the -vvv option to show the stack trace.
20:21:49  Use the --help option to see usage information.
20:21:49  
20:21:49  
20:21:49    STDERR:
20:21:49  
20:21:49  Replacing any invalid prefixes...
20:21:49  Encountered unresolvable error: <class 'sh.ErrorReturnCode_2'> - 
20:21:49  
20:21:49    RAN: /usr/bin/sed -i s/file:/file_/ data/foodon/v2024-02-21/foodon.json
20:21:49  
20:21:49    STDOUT:
20:21:49  
20:21:49  
20:21:49    STDERR:
20:21:49  /usr/bin/sed: can't read data/foodon/v2024-02-21/foodon.json: No such file or directory
20:21:49   (("\n\n  RAN: /usr/bin/sed -i s/file:/file_/ data/foodon/v2024-02-21/foodon.json\n\n  STDOUT:\n\n\n  STDERR:\n/usr/bin/sed: can't read data/foodon/v2024-02-21/foodon.json: No such file or directory\n",))

So converting the relaxed owl to obojson fails initially, and continues to do so even after the usual repairs. Try it locally.

caufieldjh commented 6 months ago

A similar thing has happened in the recent past with FOODON re: things making obojson sad: https://github.com/FoodOntology/foodon/issues/277#issuecomment-1615257724

caufieldjh commented 6 months ago

Trying out locally with ROBOT version 1.9.5:

$ wget http://purl.obolibrary.org/obo/foodon.owl
...
$ robot --vvv relax --input foodon.owl --output foodon_relaxed.owl
...
$ robot -vvv convert --input foodon_relaxed.owl --format json --output foodon.json
...
OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
For details see: http://robot.obolibrary.org/errors#obo-graph-error
java.io.IOException: errors#OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
        at org.obolibrary.robot.IOHelper.saveOntologyFile(IOHelper.java:1752)
        at org.obolibrary.robot.IOHelper.saveOntology(IOHelper.java:876)
        at org.obolibrary.robot.CommandLineHelper.maybeSaveOutput(CommandLineHelper.java:667)
        at org.obolibrary.robot.ConvertCommand.execute(ConvertCommand.java:141)
        at org.obolibrary.robot.CommandManager.executeCommand(CommandManager.java:244)
        at org.obolibrary.robot.CommandManager.execute(CommandManager.java:188)
        at org.obolibrary.robot.CommandManager.main(CommandManager.java:135)
        at org.obolibrary.robot.CommandLineInterface.main(CommandLineInterface.java:65)

The above approach is to try to repair by first removing object properties, then removing comments.

$ robot -vvv remove --input foodon_relaxed.owl --select object-properties --output foodon.json
...
2024-03-19 12:38:18,331 WARN  org.obolibrary.robot.RelatedObjectsHelper - Circular subclass definition: http://purl.obolibrary.org/obo/FOODON_00002511
2024-03-19 12:38:18,554 DEBUG org.obolibrary.robot.IOHelper - Saving ontology as OboGraphs JSON Syntax with to IRI file:/home/harry/kg-obo/foodon.json
OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
For details see: http://robot.obolibrary.org/errors#obo-graph-error
java.io.IOException: errors#OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
        at org.obolibrary.robot.IOHelper.saveOntologyFile(IOHelper.java:1752)
        at org.obolibrary.robot.IOHelper.saveOntology(IOHelper.java:876)
        at org.obolibrary.robot.CommandLineHelper.maybeSaveOutput(CommandLineHelper.java:667)
        at org.obolibrary.robot.RemoveCommand.execute(RemoveCommand.java:202)
        at org.obolibrary.robot.CommandManager.executeCommand(CommandManager.java:244)
        at org.obolibrary.robot.CommandManager.execute(CommandManager.java:188)
        at org.obolibrary.robot.CommandManager.main(CommandManager.java:135)
        at org.obolibrary.robot.CommandLineInterface.main(CommandLineInterface.java:65)
$ robot -vvv remove --input foodon_relaxed.owl --term rdfs:comment --output foodon.json
...
2024-03-19 12:40:07,351 WARN  org.obolibrary.robot.RelatedObjectsHelper - Circular subclass definition: http://purl.obolibrary.org/obo/FOODON_00002511
2024-03-19 12:40:07,587 DEBUG org.obolibrary.robot.IOHelper - Saving ontology as OboGraphs JSON Syntax with to IRI file:/home/harry/kg-obo/foodon.json
OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
For details see: http://robot.obolibrary.org/errors#obo-graph-error
java.io.IOException: errors#OBO GRAPH ERROR Could not convert ontology to OBO Graph (see https://github.com/geneontology/obographs)
        at org.obolibrary.robot.IOHelper.saveOntologyFile(IOHelper.java:1752)
        at org.obolibrary.robot.IOHelper.saveOntology(IOHelper.java:876)
        at org.obolibrary.robot.CommandLineHelper.maybeSaveOutput(CommandLineHelper.java:667)
        at org.obolibrary.robot.RemoveCommand.execute(RemoveCommand.java:202)
        at org.obolibrary.robot.CommandManager.executeCommand(CommandManager.java:244)
        at org.obolibrary.robot.CommandManager.execute(CommandManager.java:188)
        at org.obolibrary.robot.CommandManager.main(CommandManager.java:135)
        at org.obolibrary.robot.CommandLineInterface.main(CommandLineInterface.java:65)

Could that circular subclass definition be the issue? Probably not, because it's been like that since 2019 and previous KG-OBO builds (e.g., translating the 2023-05-03 version) worked as expected.

Not sure exactly what the issue is.

As a workaround, patch up the error handling so this doesn't impact the rest of the KG-OBO build.

caufieldjh commented 6 months ago

Alternatively, can default to using a base file if it's available, as it is in this case.

caufieldjh commented 6 months ago

This works:

$ robot -vvv remove --input foodon_relaxed.owl --exclude-term IAO:0000115 --output foodon.json