Annotations attached to axioms seem to disappear in Protégé

[markus-rothkoetter]

Description of the issue

When I open a file with annotations attached to axioms, they are not visible in the UI:

However, the annotations are in the corresponding ontology-file. As soon as the file is saved in any way within Protégé (unrelated from certain entities), the annotations are also lost on the file-level:

Ideas of solution

I'd be happy, if someone could test whether this issue is reproduceable. (@jannahastings, @MGlauer ? 😉) If it isn't reproduceable, it might be related to my machine.

Steps for testing:

1. Open e.g. ro-module in a text editor.
2. Check whether the annotations exist e.g. at RO_0002234
3. Open the same file in Protégé
4. Check whether the annotations exist e.g. at RO_0002234
5. Save file from Protégé
6. Check whether the annotations disappeared on file-level, too.

Other things I already checked:

• In the owlapi-repo there is no current issue explaining something like this.
• There has been such a bug in owlapi three years ago.

[l-emele]

Which exact Protégé version are you using? We had previously some problems (#471, #620) with Protégé that were related to different Protégé versions.

[markus-rothkoetter]

v5.5.0 on Win10 release-version, so there shouldn't be any problems with beta-version etc.

[jannahastings]

@markus-rothkoetter is there a feature branch with the relevant annotations committed so that I could quickly check out your files with my Protege?

[markus-rothkoetter]

@jannahastings You can select any branch, I think. Just cross-checked, if this might be related to my local branch.

Unfortunately, I can reproduce the bug on any branch I tested (including dev) Simply, open e.g. ro-module in Protégé, do an arbitrary edit (e.g. create class, change a letter in editor note or something else) and save from Protégé. Git's diff-view will show that annotations to axioms are lost in that file.

[jannahastings]

@markus-rothkoetter Does it also happen in an OWL file in RDF-XML format? Or is it just the Manchester syntax files? Never mind I realise that the ro-module is in RDF-XML already :-/

[jannahastings]

@markus-rothkoetter I can save the file from Protege without losing annotations on axioms

[jannahastings]

@markus-rothkoetter Did you try just with a simple axiom annotation - just text no URLs?

[markus-rothkoetter]

@jannahastings

Did you try just with a simple axiom annotation - just text no URLs?

The strange thing is that the former annotations aren't visible at all, but new ones show. I'll try with a new vanilla ontology file, if the issue is reproduceable.

[markus-rothkoetter]

We can confirm that this is a bug in Protégé that is reproduceable on different machines with different OS, this means:

Annotations to axioms (InverseOf, SubPropertyOf, potentially more) are lost, when a file is opened and saved from Protégé. This might have resulted in data loss of term tracker item-annotations already and will lead to further data-loss (inevitablely). The actual ontology axioms are NOT affected and therefore save.

The next steps are planned as followed:

• Reaching out to the Protégé/OWL-api developers asking for a fix/patch.

• Announcing to the oeo-dev team not to use the axiom annotations feature, but placing all annotations as class annotations only.

• Estimating the extent of the feature usage and a potential workaround/hot-fix to prevent further immediate data-loss

• Pushing this to the next OEO-DEV-meeting, so that we can discuss whether a data-retrival of potential losses is necessary/feasible.

[markus-rothkoetter]

During #1197 it seemed like owlapi fills arbitrary axioms at places where they were already lost. Inspect at least at RO_0004009, RO_0002505 and is defined by

[markus-rothkoetter]

There seem to be to different syntax uses for annotating axioms: Syntax 01:

Syntax 02:

Maybe this is dependend on the kind of saving format.

AFAIK, there are only axioms affected which use syntax 02

Currently, those are not much:

• No. 1 + 2 are custom ones for the term tracker item
• No. 3 + 4 are from the source ontology, I'd judge from a quick look.

Ergo, the problem seems to have a quite limited scope. @jannahastings Syntax 02 is RDF-XML, isn't it? So, is syntax 01 pure Manchester-Syntax then?

[l-emele]

From OEO DEV Meeting 39 (decisions highlighted):

• Short explanation of bug + steps to counter it
  • Two approaches to add annotations: at class level or at axiom level
  • Adding at axiom level results in the axioms being lost on the subsequent save of the file (not the first save)
  • Two things to decide on: how to proceed in the meantime until the bug is fixed, and how do we treat potential lost data in the past.
• Estimation of extent of effects (aka how often is this feature currently used)
  • Bug seems to relate only to files saved in RDF-XML (i.e. mainly imports modules)
  • There seem to be fewer than 15 of these.
  • Workaround for these modules will be to attach annotations to the class/object property directly rather than the axiom
  • Alternative option "proceed with caution"
  • Question: is it a problem with Protege really or with switching branches while protege is open? Workflow: close Protege, switch branch, open Protege... This avoids other problems, but this particular problem is reproducible without switching branches.
  • Would a different format work better? RDF-XML vs. Manchester syntax. But, general class axioms can't be saved in Manchester syntax.
  • There are no guidelines about at which level to add the annotations. Should we have a general agreement to always use class-level annotations? Warning that tools for browsing ontologies likely do not show axiom annotations.
  • Additional guidance would be nice about saying in the annotation what happened in the change, which for example Lukas always does.
  • Proposal: move all the currently affected axiom annotations to class-level annotations. (agreed)
• Discussion concerning treatment of potential data already lost (cost-beenfit of retrival)
  • Should we look carefully at the Git commit history in order to find any potentially lost annotations?
  • Not needed because even accidental changes would have been seen in the pull request. Several such issues occurred in the past and were picked up on peer review.

[stap-m]

Please take over this issue @stage1407

[stap-m]

@stage1407 Please add here the next steps we decided on in our meeting yesterday. Please also record your work on the issue here by commenting. Thanks!

[stage1407]

The next steps are:

1. Setting up a minimal ontology.
2. Reproduce the bug on this ontology
3. Send it to the Protégé developers

[stage1407]

I noticed an interesting and strange behaviour, when reading the original file and the file to be overwritten. All axioms that were annotated by rdfs:comment were exactly the ones that were not discarded.

This allows a new workaround for us and provides interesting information for the developers of Protege.

[stage1407]

Here a little example ontology, if anyone wants to try it on their own machine.

Protege will add some lines automatically, but these should be irrelevant. (In detail: It infers from the context that Property must be an Individual and an Annotations.)

<?xml version="1.0"?> <rdf:RDF xmlns="http://www.semanticweb.org/alexander.stage/ontologies/2022/9/untitled-ontology-13#" xml:base="http://www.semanticweb.org/alexander.stage/ontologies/2022/9/untitled-ontology-13" xmlns:owl="http://www.w3.org/2002/07/owl#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:xml="http://www.w3.org/XML/1998/namespace" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"> <owl:Ontology rdf:about="http://www.semanticweb.org/alexander.stage/ontologies/2022/9/untitled-ontology-13"/>

<!-- /////////////////////////////////////////////////////////////////////////////////////// // // Object Properties // /////////////////////////////////////////////////////////////////////////////////////// -->

<!-- http://example.com/ -->

<owl:ObjectProperty rdf:about="http://example.com/Property"> <rdfs:label xml:lang="en">Property</rdfs:label> </owl:ObjectProperty> <owl:Axiom> <owl:annotatedSource rdf:resource="http://example.com/Property"/> <owl:annotatedProperty rdf:resource="http://example.com/Property"/> <owl:annotatedTarget rdf:resource="http://example.com/Property"/>

<!-- Here can be written more and it disappears --> </owl:Axiom> </rdf:RDF>

[stage1407]

I added these lines below and they confirm my assumption, with rdfs:label from my second comment.

<owl:ObjectProperty rdf:about="http://example.com/Property2"> <rdfs:label xml:lang="en"> Property </rdfs:label> </owl:ObjectProperty> <owl:Axiom> <owl:annotatedSource rdf:resource="http://example.com/Property"/> <owl:annotatedProperty rdf:resource="http://example.com/Property"/> <owl:annotatedTarget rdf:resource="http://example.com/Property"/> <rdfs:comment> Yippee, I still exist </rdfs:comment> </owl:Axiom>

[fabianneuhaus]

After a short discussion Alexander and I agreed on the following next steps:

• Alexander tries to open the minimal example ontology with the OWL API (instead of Protege) and sees whether he can reproduce it. If so, he sends a bug report to https://github.com/owlcs/owlapi
• One strategy to avoid the problem is to add annotations to the axioms (e.g., "This axiom was automatically extracted from the ontology X". Thus, in a first step we try to use the OWL API to add these axioms to the axioms in the modules. If that fails, we might need to hack something, but that seems less preferable.

[stage1407]

"One strategy to avoid the problem is to add annotations to the axioms [...]"

I think this should work, but I tested this only with the Annotation Property: rdfs:comment. So, with rdfs:comment you are on the save side.

[stage1407]

Report: I read the OWL API Introduction, Documentation and Tutorial. Then I tried to include the OWL API in a Java test project. Unfortunately it seems to have problems to include this specific library. (Cannot resolve import) Then I tried it in the IDEs: Visual Studio Code and Eclipse.

Next try would be via IntelliJ, as this offers more detailed ways to include libraries.

I also went through the issues of the OWL API on Github. The issue we are having is not documented there.

Next steps:

• Try to create a test project via IntelliJ.
• Maybe just send the created bug report to the Protege devs.

[fabianneuhaus]

If you have problem to use the OWL API, you could ask Simon Flügel who used it recently for his BA thesis. His code is available here, if you want to have a look https://github.com/gavel-tool/python-gavel-owl/tree/dev/java/src

(Part of the system was developed by Anna Kleinau.)

[stage1407]

The OWL API is now correctly integrated into a test project.

[stage1407]

The next step will be to write a test project to find out if the error is caused by Protege or the OWL API.

[stage1407]

Now, I've written a test project and got a strange exception.

NoClassDefFoundException: com/google/inject/Provider

Thereupon I tried a little bit back and forth, but without success. Now I contacted Simon. Maybe he has some ideas about this.

[stage1407]

I have found a jar of the OWL API 5.1.5 (via jar-downloader) But the apibinding package is missing, which I need to use the OWLManager. A little bit confusing, because the current documented version is the 4.5.19.

Tomorrow I will try to find out, whether I can patch this somehow or I have to setup a whole MavenProject for it..

[stap-m]

@stage1407 did you try out the script of @sfluegel05 ?

[stage1407]

@stap-m I could copy his MavenProject, but I don't know about side-effects but less effort than my ideas. I will try that.

[stage1407]

That's it! Now we have results.

The OWL API changes some things in the ontology, but no axioms or other information disappear.

The disappearance of axioms seems to be triggered by Protege.

[stage1407]

Input of the first Test:

Output of the first Test:

[stage1407]

Next steps:

1. design a test example of a more realistic ontology in Manchester syntax. Whereby it should no longer be about such special triples like (p p p .) and the example should be closer to the actual bug .

2. add a comment to each axiom, what it means in the natural language (maybe a separate issue for that?)

3. if comment is missing for an axiom, let the build tests of the ontology report warnings in Github or fail directly (maybe a separate issue for that, too..)

[stage1407]

TODO:

• send bug report to protege devs

• if the import module process from pr #1268 solves the problem, then we are done with this issue, otherwise i will write a script, that adds a comment automatically to each axiom, which isnt commented yet. (+ documentation of this process)

[stage1407]

The way I read the discussion, it's a similar problem, but it won't really solve this problem because the point here is to add a comment to each axiom so it doesn't disappear. AFAIK this does not happen for each import But my last stand was that all axioms are imported. @stap-m do you know where these axioms are imported from, and by which processes? Then maybe only an already existing script would have to be adjusted a bit, and we would be done.

Otherwise I have to figure it out first.

[stage1407]

Robot does not provide a direct command to add comments to a dynamic set of axioms, but it is possible to write a program using Robot commands, reading the imported axioms, and creating the comments depending on the axiom in question The alternative would be to program with the OWL API

So we have the following options

• Option 1: Call Robot in a script that can do such things

• Option 2: write a program using the OWL API

I would prefer the first option, since it requires less effort and we would get the same result as with the second option.

[stap-m]

@stap-m do you know where these axioms are imported from, and by which processes? Then maybe only an already existing script would have to be adjusted a bit, and we would be done.

This is the import script for ro-extracted.owl.

[stap-m]

@stage1407 PR #1452 also deals with the extraction script. Please wait for it to be merged.

[jamesaoverton]

A linked issue on the ROBOT tracker led me here. You are definitely experiencing some strange behaviour. I work with OWL annotation axioms every day and do not see them disappear. I would guess that there's a problem with a namespace or strange URL, or with imports. Whatever the underlying problem, adding annotations to every imported axiom can't be the right solution.

If you have a minimal example, I will look at it. The example in comment https://github.com/OpenEnergyPlatform/ontology/issues/1179#issuecomment-1328786054 is not a good example, because it is malformed at line 30: the OWL axiom annotation does not have actual annotation content, so OWLAPI is dropping the annotated* reification triples.

[matentzn]

Typical cause of the problem: No Declaration axiom from the annotation property used in the axiom annotation.