UCO support for Protégé

[ajnelson-nist]

Disclaimer

Participation by NIST in the creation of the documentation of mentioned software is not intended to imply a recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that any specific software is necessarily the best available for the purpose.

Background

Protégé is a tool for editing ontologies.

https://protege.stanford.edu/

It is able to open and interact with ontologies stored as local files in a user's desktop environment. It is also capable of resolving all of the ontology imports (encoded as owl:import statements), recursively retrieving all ontologies referenced from any loaded ontology.

By default, Protégé will do an over-the-wire retrieval on encountering an owl:imports statement: The referenced IRI will be downloaded in whatever RDF serialization is offered (seemingly preferring application/rdf+xml). It is possible to include an "Override" XML file that can be interpreted as: "Whenever Protégé encounters this IRI, instead of loading a file from a network retrieval, load a file from this hard-coded relative or absolute path."

There is a slight technical matter with the XML file: It must reside in the same directory as the ontology file one would open with Protégé.

Requirements

Requirement 1

UCO should store a Protégé catalog-v001.xml file in ontology/uco/master/, enumerating all UCO ontology files.

Requirement 2

The catalog-v001.xml file's hard-coded enumeration must be tested to be in sync. with UCO's imports.

Requirement 3

CASE must provide the same Protégé support as UCO, maintaining its own catalog-v001.xml file in ontology/master/, enumerating all CASE and UCO ontology files. While this might seem out of scope of UCO's purview, this requirement is also to ensure UCO can enable any downstream ontology to provide the same support for Protégé that UCO does.

Risk / Benefit analysis

Benefits

• Storage of this file would enable anybody loading all of UCO into Protégé to navigate the class hierarchy without requiring network access.
  • Similarly, any pre-release version of UCO can also be loaded into Protégé, prevent conflicts with ontology files that are on the web as part of the most recent release.

Risks

• The catalog-v001.xml file is necessarily a hard-coded enumeration of resources. Hence, any additions of new .ttl files would need to be kept in sync to maintain referential integrity within the Protégé application.
  • This is believed to be a not-terribly-difficult piece of custom programming to implement as a unit test. However, it is custom software that would add to the ontology-level software base.
• It's fair to consider that each ontology file should get a catalog-v001.xml in order to locally resolve its imports. Else, Protégé is only compatible with UCO when loaded in its entirely. This would be additional work to maintain.
• The import statements in UCO will potentially use owl:versionIRI. If so, these catalog XML files would need to be updated with every release that changes the owl:versionIRI statement.
• CASE currently tracks UCO as a Git submodule. Directions would need to be added to CASE's documentation---likely in CONTRIBUTE.md---that Protégé local resolution would only work if git submodule update --init has been run at least once.

Competencies demonstrated

Competency 1

A user is interested in using Protégé to load all of UCO's current state in the develop branch, which has some changes implemented since the last UCO release.

Competency Question 1.1

How does the user see the current version of observable:File in the develop branch?

Result 1.1

If there is only one catalog-v001.xml in UCO - uco.ttl would need to be opened with Protégé. Then, observable:File's current state would be viewable through the class navigator.

If instead each ontology directory gets a catalog-v001.xml - observable.ttl would need to be opened. The rest is as above.

Solution suggestion

• Add catalog-v001.xml alongside uco.ttl, in the directory ${top_srcdir}/ontology/uco/master/.
• Add a program (likely Python) that confirms these sets of IRIs align:
  • For an ontology file X, the set of IRIs in that file's transitive closure (which might only be apparent from the monolithic test build);
  • For the same ontology file X, the set of IRIs mapped in the catalog-v001.xml.
• Confirm the test program developed for UCO functions for CASE, and thus in theory other downstream adopters.

Coordination

• Tracking in Jira ticket OC-261
• [x] Requirements to be discussed in OC meeting, 2022-08-16
• [x] Requirements Review vote occurred, passing, on 2022-08-16
• [x] Requirements development phase completed.
• [x] Solution announced to OCs on 2023-03-15
• [x] Solutions Approval to be discussed in OC meeting, 2023-03-23
• [x] Solutions Approval vote occurred, passing, on 2023-03-23
• [x] Solutions development phase completed.
• [x] Backwards-compatible implementation merged into develop for the next release
• [x] develop state with backwards-compatible implementation merged into develop-2.0.0
• [x] Backwards-incompatible implementation merged into develop-2.0.0 (N/A)
• [x] Backwards-compatible CASE implementation merged into develop for the next release
• [x] Milestone linked
• [x] Documentation logged in pending UCO release page
• [x] Documentation logged in pending CASE release page

[ajnelson-nist]

PR 450 has been filed to start the implementation for this proposal. Thanks again, @DrSnowbird, for contributing the start of this branch.

Unfortunately, a curious issue arose with trying to load the Collections Ontology shape file. There's a chance it will be resolvable with an extra Git submodule based interaction - I'm out of time to test tonight.

My current feeling is that the testing infrastructure complexity for this might be too high for integration with the UCO 1.0.0 release, but it is a backwards-compatible change that could be integrated with any 1.x.0.

[ajnelson-nist]

PR 450 now has an implemented solution for this Issue. The summary of effects is that with the associated PRs merged, UCO works in Protégé wholly with locally-stored (/-versioned) ontology files, and the generation mechanism is confirmed to work for downstream ontologies. I have tested this with CASE and CASE-Corpora (test links are in PR 450). CASE-Corpora is now able to generate catalog-v001.xml files that cover its ontology import closure.

Some unexpected developments occurred:

• The build order no longer made sense with the Python virtual environment being nestled under /tests, due to /ontology needing Python to have access to rdflib. That environment is now built in the repository root, for UCO and for CASE. Rationale is described in 66d0c38.
• The catalog-generating script is designed to create one catalog-v001.xml file per directory with ontology files. This is to mirror the behavior of Protégé, where that XML file has an effect just on graph files in its current directory, and it does not seem to be transitive (i.e. Protégé does not seem to inherit catalog-v001.xml files when following relative-pathed uri references). This is also necessary because file references are only done as relative paths from the XML file's housing directory.
• The catalog-generating script currently generates an XML file that lists paths to backing files for every imported ontology-reference (that is, the object of every $subject owl:imports $object triple) -- "every" meaning through the transitive import closure of whatever ontology file was passed to the script as the "Focus ontology". The hard-coding for this name resolution is done in a few tab-separated value files. The catalog files regenerate as part of CI; see the Makefile reivisions for the recipes and /etc for the new files, in UCO, CASE, and CASE-Corpora.

The overall increase in risk is for projects that track UCO and CASE as Git submodules for the sake of re-using their virtual environment and/or monolithic ontology build. The virtual environment motion means some tracking projects will need to update paths to scripts. I'm aware that this will impact the CASE Python Utilities' monolithic build tracking and the documentation engines (CASE's, UCO's) most, but I suggest that overall this is logistically acceptable, as paths only need to be updated once per tracking Git project.