Bug: Build freezes on `prepare_release` (`mondo-ingest-edit.owl-obo-report.tsv`)

Overview

I'm running the build PR #601. However it has frozen 3 times on the same step.

Background

The first two times I ran it, I accidentally closed my macbook and put it to sleep. The first time, myself, the second time, I had the screen at a 45 degree angle, and later it was closed, so I assume it was my cats. Anyway, I thought that it going to sleep might have caused the issue.

I ran it a third time though; been running for 8+ hours, and has not went to sleep. It's using CPU, but IDK what it's doing. I assume it's stuck in an infinite loop.

Screenshot: Docker Stats tab

Screenshot 2024-07-02 at 11 25 54 PM

Each time, it freezes on this command:

robot --catalog catalog-v001.xml report -i tmp/merged-mondo-ingest-edit.owl  --profile profile.txt --fail-on None --print 5 -o reports/mondo-ingest-edit.owl-obo-report.tsv

This is the goal for reports/mondo-ingest-edit.owl-obo-report.tsv, which runs at the end of the build as part of prepare_release.

Logs

Just for comparison, here's a snippet from the log before and after this goal runs, taken from my recent build on the "intelligent refresh" PR.

Log from different build for comparison

``` # intelligent refresh context robot --catalog catalog-v001.xml template \ --template external/nando-mappings.robot.tsv \ --prefix "orcid: https://orcid.org/" \ annotate \ --ontology-iri http://purl.obolibrary.org/obo/mondo/external/nord.robot.owl \ --version-iri http://purl.obolibrary.org/obo/mondo/external/2024-06-26/nord.robot.owl \ -o external/nando-mappings.robot.owl robot --catalog catalog-v001.xml template \ --template external/ordo-subsets.robot.tsv \ --prefix "orcid: https://orcid.org/" \ annotate \ --ontology-iri http://purl.obolibrary.org/obo/mondo/external/nord.robot.owl \ --version-iri http://purl.obolibrary.org/obo/mondo/external/2024-06-26/nord.robot.owl \ -o external/ordo-subsets.robot.owl ODK Makefile v1.5-dev ODK Image v1.5-dev ROBOT version 1.9.5 Ammonite REPL & Script-Runner, 2.5.9 DOSDP-Tools v0.19.3 Konclude v0.7.0-1138 Jena v4.9.0 Relation-Graph v2.3.2 Ontology Access Kit v0.6.6 Repository is up-to-date. robot --catalog catalog-v001.xml reason --input tmp/mondo-ingest-preprocess.owl --reasoner ELK --equivalent-classes-allowed asserted-only \ --exclude-tautologies structural --output test.owl && rm test.owl robot --catalog catalog-v001.xml verify -i tmp/mondo-ingest-preprocess.owl --queries ../sparql/properties-as-annotation-and-object-violation.sparql ../sparql/ordo-mapping-annotations-violation.sparql -O reports PASS Rule ../sparql/properties-as-annotation-and-object-violation.sparql: 0 violation(s) PASS Rule ../sparql/ordo-mapping-annotations-violation.sparql: 0 violation(s) robot --catalog catalog-v001.xml report -i tmp/merged-mondo-ingest-edit.owl --profile profile.txt --fail-on None --print 5 -o reports/mondo-ingest-edit.owl-obo-report.tsv Violations: 593254 ----------------- ERROR: 60708 WARN: 503787 INFO: 28759 First 5 violations: ERROR deprecated_class_reference http://id.who.int/icd/entity/1000312374 rdfs:subClassOf http://id.who.int/icd/entity/1345372703 ERROR deprecated_class_reference http://id.who.int/icd/entity/1001085090 rdfs:subClassOf http://id.who.int/icd/entity/583310882 ERROR deprecated_class_reference http://id.who.int/icd/entity/1002125483 rdfs:subClassOf http://id.who.int/icd/entity/794078271 ERROR deprecated_class_reference http://id.who.int/icd/entity/1002411765 rdfs:subClassOf http://id.who.int/icd/entity/309514703 ERROR deprecated_class_reference http://id.who.int/icd/entity/1002515350 rdfs:subClassOf http://id.who.int/icd/entity/460984650 robot --catalog catalog-v001.xml merge -i mondo-ingest.owl convert -f ofn -o tmp/validate.ofn ```

ODK versions

All these builds ran on ODK latest. It is up to date. From what I can see, it hasn't changed in about a month.

ROBOT versions

I see that during prepare_release, it prints out various versions of software. ODK latest is currently on ROBOT version 1.9.5, but dev is on 1.9.6, so I wonder if 1.9.5 has some bug relevant here which 1.9.6 has fixed.

Things tried, but failed

Changing ODK to `dev`

I just realized that we recently changed ODK from dev to latest on develop:

579

We have not yet done that on main (#580).

I do believe that this is the first build that I / we have run since then. All the builds I've done in the last month and I'm not sure how much further back have been on dev.

So I'm going to run this on ODK dev now and see how that does.

edit: Update: tried this and didn't work.

Possible cause & solution

QC check causes crazy / endless loop

Nico:

I deactived 99% of the report check in mondo ingest, as it is totally redundant. Try again running the build for https://github.com/monarch-initiative/mondo-ingest/pull/601

Nico (slack):

there is a QC check that looks for overlap between exact synonyms and label. Could be that the respective sparql query just became method because all labels now had an exact synonym with the same name

monarch-initiative / mondo-ingest