search

ncbo / ncbo_cron

Jobs that run on a regular basis in the NCBO infrastructure
Other
2 stars 6 forks source link

Clean up repeated failures during nightly ontology pull process #58

Open jvendetti opened 1 year ago

jvendetti commented 1 year ago

Over the last several days I had to look at the scheduler-pull.log file (/srv/ncbo/ncbo_cron/logs/scheduler-pull.log) as part of a troubleshooting process for CADSR-VS. Examining this log file is cumbersome because we have a number of ontologies that are generating the same errors every night. It would be nice to set aside some time to clean them up.

Full report of the problematic ontologies below, organized by error type.

Duplicate naming errors

APO

E, [2022-08-01T18:01:15.226668 #31664] ERROR -- : Unable to create a new submission in OntologyPull: {:proc_naming=>{:duplicate=>"There is already a persistent resource with id http://data.bioontology.org/ontologies/APO/submissions/33"}}

ARO

E, [2022-08-01T18:01:22.742070 #31664] ERROR -- : Unable to create a new submission in OntologyPull: {:proc_naming=>{:duplicate=>"There is already a persistent resource with id http://data.bioontology.org/ontologies/ARO/submissions/17"}}

CDAO

E, [2022-08-01T18:02:34.886460 #31664] ERROR -- : Unable to create a new submission in OntologyPull: {:proc_naming=>{:duplicate=>"There is already a persistent resource with id http://data.bioontology.org/ontologies/CDAO/submissions/8"}}

CMPO

E, [2022-08-01T18:03:11.318392 #31664] ERROR -- : Unable to create a new submission in OntologyPull: {:proc_naming=>{:duplicate=>"There is already a persistent resource with id http://data.bioontology.org/ontologies/CMPO/submissions/15"}}

LABO

E, [2022-08-01T18:07:18.877569 #31664] ERROR -- : Unable to create a new submission in OntologyPull: {:proc_naming=>{:duplicate=>"There is already a persistent resource with id http://data.bioontology.org/ontologies/LABO/submissions/5"}}

LC-MEDIA

E, [2022-08-01T18:07:20.705242 #31664] ERROR -- : Unable to create a new submission in OntologyPull: {:proc_naming=>{:duplicate=>"There is already a persistent resource with id http://data.bioontology.org/ontologies/LC-MEDIA/submissions/18"}}

Bad pull URLs

ABD

I, [2022-08-03T18:01:00.295924 #27383] INFO -- : RemoteFileException: No submission file at pull location http://brd.bsvgateway.org/disease.owlrdf.xml for ontology ABD.

BCI-O

E, [2022-08-01T18:01:29.709121 #31664] ERROR -- : Problem retrieving BCI-O in OntologyPull: Failed to open TCP connection to bci.pet.cs.nctu.edu.tw:80 (getaddrinfo: Name or service not known)

CCONT

E, [2022-08-01T18:02:32.485678 #31664] ERROR -- : Problem retrieving CCONT in OntologyPull: /offline/: 404

DCCDFV

E, [2022-08-01T18:03:24.242683 #31664] ERROR -- : Problem retrieving DCCDFV in OntologyPull: Failed to open TCP connection to :80 (No route to host - connect(2) for nil port 80)

DCO

I, [2022-08-01T18:03:24.655898 #31664] INFO -- : RemoteFileException: No submission file at pull location http://file.dispedia.de/dispediaCoreComplete.xml for ontology DCO.

EXO

I, [2022-08-03T18:11:45.920876 #27383] INFO -- : RemoteFileException: No submission file at pull location https://raw.githubusercontent.com/CTDbase/exposure-ontology-draft/master/src/ontology/exo.obo for ontology EXO.

FIDEO

I, [2022-08-03T18:11:53.822210 #27383] INFO -- : RemoteFileException: No submission file at pull location https://gitub.u-bordeaux.fr/erias/fideo/-/raw/master/fideo_core.owl for ontology FIDEO.

GML

I, [2022-08-03T18:12:10.795091 #27383] INFO -- : RemoteFileException: No submission file at pull location https://www.seegrid.csiro.au/subversion/CGI_CDTGVocabulary/trunk/OwlWork/ogc-gml.owl for ontology GML.

GPML

E, [2022-08-01T18:05:26.436922 #31664] ERROR -- : Problem retrieving GPML in OntologyPull: /convert/rdfa/xml/http%3A%2F%2Fvocabularies.wikipathways.org%2Fgpml: 500

HRDO

I, [2022-08-03T18:13:43.764140 #27383] INFO -- : RemoteFileException: No submission file at pull location http://ics.upmc.fr/hrdo/hrdo.owl for ontology HRDO.

ISO19115

I, [2022-08-01T18:07:10.139023 #31664] INFO -- : RemoteFileException: No submission file at pull location https://www.seegrid.csiro.au/subversion/xmml/metadata/ISO19115/iso-19115.owl for ontology ISO19115.

ISO19115EX

I, [2022-08-01T18:07:10.689517 #31664] INFO -- : RemoteFileException: No submission file at pull location https://github.com/ISO-TC211/GOM/blob/master/isotc211_GOM_harmonizedOntology/19115-1/2014/iso19115-1ExtentInformation.owl for ontology ISO19115EX.

ISO19115MI

I, [2022-08-01T18:07:11.131932 #31664] INFO -- : RemoteFileException: No submission file at pull location https://raw.githubusercontent.com/ISO-TC211/GOM/master/isotc211_GOM_harmonizedOntology/19115-1/2014/iso19115-1MetadataInformation.ttl for ontology ISO19115MI.

ISO19115SRS

I, [2022-08-01T18:07:11.546271 #31664] INFO -- : RemoteFileException: No submission file at pull location https://raw.githubusercontent.com/ISO-TC211/GOM/master/isotc211_GOM_harmonizedOntology/19115-1/2014/iso19115-1ReferenceSystemInformation.ttl for ontology ISO19115SRS.

ISO639-2

I, [2022-08-01T18:07:12.054122 #31664] INFO -- : RemoteFileException: No submission file at pull location http://aims.fao.org/aos/languagecode.owl for ontology ISO639-2.

LCTGM

E, [2022-08-01T18:07:21.081990 #31664] ERROR -- : Problem retrieving LCTGM in OntologyPull: /static/data/vocabularygraphicMaterials.rdf.both.zip: 404

LEGALAPA

I, [2022-08-01T18:07:21.538074 #31664] INFO -- : RemoteFileException: No submission file at pull location http://alexandergarcia.name/ontos/legal/legal.owl for ontology LEGALAPA.

LIPRO

I, [2022-08-01T18:09:24.071174 #31664] INFO -- : RemoteFileException: No submission file at pull location http://cbakerlab.unbsj.ca:8080/lipids/lipid-classification-service-ontology.owl for ontology LIPRO.

MIRNAO

I, [2022-08-03T18:16:23.658709 #27383] INFO -- : RemoteFileException: No submission file at pull location http://mirna-ontology.googlecode.com/svn/trunk/src/ontology/mirnao.owl for ontology MIRNAO.

OCDM

E, [2022-08-01T18:12:24.818415 #31664] ERROR -- : Problem retrieving OCDM in OntologyPull: /share/downloads/ocdm/release/latest/uncompressed//pun_ocdm.owl: 404

OGI

E, [2022-08-01T18:13:12.697497 #31664] ERROR -- : Problem retrieving OGI in OntologyPull: /svn/trunk/src/OGI.owl: 404

PANET_DEV

E, [2022-08-01T18:14:39.170602 #31664] ERROR -- : Problem retrieving PANET_DEV in OntologyPull: /ExPaNDS-eu/ExPaNDS-experimental-techniques-ontology/raw/onto/source/PaNET.owl: 404

PMO-SPEED

I, [2022-08-03T18:19:41.338086 #27383] INFO -- : RemoteFileException: No submission file at pull location https://raw.githubusercontent.com/LD4P/PerformedMusicOntology/master/Vocabularies/PMOPlayingSpeed.rdf for ontology PMO-SPEED.

PROPREO

I, [2022-08-03T18:20:51.993589 #27383] INFO -- : RemoteFileException: No submission file at pull location http://lsdis.cs.uga.edu/projects/glycomics/propreo/ProPreO-060506.owl for ontology PROPREO.

QUDT2

E, [2022-08-01T18:16:37.215297 #31664] ERROR -- : Problem retrieving QUDT2 in OntologyPull: /2.0/schema/SCHEMA_QUDT-v2.0.ttl: 404

RB

I, [2022-08-03T18:21:09.376835 #27383] INFO -- : RemoteFileException: No submission file at pull location https://regenbase.cs.miami.edu/ontologies/regenbase_vocabulary.owl for ontology RB.

RDA-ISSUANCE

I, [2022-08-03T18:21:10.412531 #27383] INFO -- : RemoteFileException: No submission file at pull location http://rdaregistry.info/termList/ModeIssue for ontology RDA-ISSUANCE.

RH-MESH

I, [2022-08-03T18:21:22.292164 #27383] INFO -- : RemoteFileException: No submission file at pull location http://phenomebrowser.net/ontologies/mesh/mesh.owl for ontology RH-MESH.

SCIO

E, [2022-08-01T18:17:00.866991 #31664] ERROR -- : Problem retrieving SCIO in OntologyPull: /scio/SCIO_51.owl: 404

SD3

E, [2022-08-01T18:17:02.139689 #31664] ERROR -- : Problem retrieving SD3 in OntologyPull: /ontologies/SimulationScenarioDeviations.owl: 404

VSO

E, [2022-08-01T18:17:55.889432 #31664] ERROR -- : Problem retrieving VSO in OntologyPull: /svn/releases/2012-4-25/vso.owl: 404

Unparseable

CCON

E, [2022-08-01T18:02:30.585463 #31664] ERROR -- : The new file for ontology CCON, submission id: 9 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/CCON/9/owlapi.xrdf cannot be found.

CHEMBIO

E, [2022-08-01T18:02:51.635225 #31664] ERROR -- : The new file for ontology CHEMBIO, submission id: 11 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/CHEMBIO/11/owlapi.xrdf cannot be found.

CYAN

E, [2022-08-01T18:03:22.888703 #31664] ERROR -- : The new file for ontology CYAN, submission id: 2 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/CYAN/2/owlapi.xrdf cannot be found.

ERO

E, [2022-08-01T18:04:47.862983 #31664] ERROR -- : The new file for ontology ERO, submission id: 15 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/ERO/15/owlapi.xrdf cannot be found.

FIRE

E, [2022-08-01T18:04:58.996372 #31664] ERROR -- : The new file for ontology FIRE, submission id: 8 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/FIRE/8/owlapi.xrdf cannot be found.

FPLX

E, [2022-08-01T18:05:05.428481 #31664] ERROR -- : The new file for ontology FPLX, submission id: 39 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/FPLX/39/owlapi.xrdf cannot be found.

GRO-CPGA

E, [2022-08-01T18:05:28.985646 #31664] ERROR -- : The new file for ontology GRO-CPGA, submission id: 12 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/GRO-CPGA/12/owlapi.xrdf cannot be found.

GVP

E, [2022-08-01T18:05:32.840863 #31664] ERROR -- : The new file for ontology GVP, submission id: 8 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/GVP/8/owlapi.xrdf cannot be found.

HSO

E, [2022-08-01T18:06:53.000400 #31664] ERROR -- : The new file for ontology HSO, submission id: 10 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/HSO/10/owlapi.xrdf cannot be found.

IAML-MOP

E, [2022-08-01T18:06:56.776103 #31664] ERROR -- : The new file for ontology IAML-MOP, submission id: 3 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/IAML-MOP/3/owlapi.xrdf cannot be found.

PAE

E, [2022-08-01T18:14:38.277605 #31664] ERROR -- : The new file for ontology PAE, submission id: 14 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/PAE/14/owlapi.xrdf cannot be found.

PSDS

E, [2022-08-01T18:16:30.993178 #31664] ERROR -- : The new file for ontology PSDS, submission id: 7 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/PSDS/7/owlapi.xrdf cannot be found.

SOY

E, [2022-08-01T18:17:10.371512 #31664] ERROR -- : The new file for ontology SOY, submission id: 3 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/SOY/3/owlapi.xrdf cannot be found.

WIKIPATHWAYS

E, [2022-08-01T18:18:09.613567 #31664] ERROR -- : The new file for ontology WIKIPATHWAYS, submission id: 227 did not clear OWLAPI: LinkedData::Parser::OWLAPIParserException: OWLAPI java command exited with 0. Output file /srv/ncbo/repository/WIKIPATHWAYS/227/owlapi.xrdf cannot be found.

alexskr commented 1 year ago

DCO - is a retired ontology that is also marked as metadata-only ontology so pull location should be disabled.