ontodev / robot

ROBOT is an OBO Tool
http://robot.obolibrary.org
BSD 3-Clause "New" or "Revised" License
258 stars 70 forks source link

Unclear expressivity results from robot measure #1206

Open bparsia opened 2 weeks ago

bparsia commented 2 weeks ago

I'm getting weird expressivity results using measure. Here's the results of 10 files fro mthe MOWL corpus:

expressivity single_value expressivity_incl single_value expressivity + single_value expressivity_incl + single_value expressivity single_value expressivity_incl single_value expressivity single_value expressivity_incl single_value expressivity RRESTREH+IF(D) single_value expressivity_incl RRESTREH+IF(D) single_value expressivity RRESTR(D) single_value expressivity_incl RRESTR(D) single_value expressivity RRESTRCCINTEHIQ single_value expressivity_incl RRESTRCCINTEHIQ single_value expressivity single_value expressivity_incl single_value expressivity RRESTRCUEROIN(D) single_value expressivity_incl RRESTRCUEROIN(D) single_value

it happens with --metrics extended and all. I think maybe some internal variable names are not getting properly interpolated?

matentzn commented 2 weeks ago

@bparsia whaaaaat an amazing surprise to see you here :D I assume its you, Bijan!

Although ROBOT is simply wrapping the OWL API, I can take a look.

Here is an example that reproduces the problem:

robot measure -I http://purl.obolibrary.org/obo/ro.owl  --format json --metrics all --output ro-metrics.json

1.9.6:

   "expressivity": "RRESTRCUCINTUNIVRESTREROIF(D)",
    "expressivity_incl": "RRESTRCUCINTUNIVRESTREROIF(D)",

ROBOT wraps the OWLAPI DLExpressivity class:

private String getExpressivity(boolean included) {
    DLExpressivity dl = new DLExpressivity(getOntology());
    dl.setImportsClosureUsed(included);
    dl.setOntology(getOntology());
    return dl.getValue();
  }

@ignazio1977 Is there anything that you can think off that might have caused this?

bparsia commented 2 weeks ago

Yes it's me :) I'm trying to use robot for gathering stats from collections before doing some traversal experiments ;)

I'll also note that afaict there's no way to get results to stout, which means I'm doing some file hacking to get the results, fwiw.

Looking: https://github.com/ontodev/robot/blob/master/robot-core/src/main/java/org/obolibrary/robot/metrics/OntologyMetrics.java

getExpressivity() just uses the DLExpressivty metric but maybe you need to use https://owlcs.github.io/owlapi/apidocs_4/org/semanticweb/owlapi/util/DLExpressivityChecker.html

esp getDescriptionLogicName?

matentzn commented 2 weeks ago

I tried it:

https://github.com/ontodev/robot/pull/1207

It does not work, still the same:

    "expressivity": "RRESTRCUCINTUNIVRESTREROIF(D)",
    "expressivity_incl": "RRESTRCUCINTUNIVRESTREROIF(D)",
ckindermann commented 2 weeks ago

Hi both! :) I proposed a workaround in #1207

matentzn commented 2 weeks ago

@ckindermann your idea is good, but I am not 100% sure that the languages that come out of this process are very recognisable:

"expressivity": "RRESTRCUCINTUNIVRESTREROIF(D)",
"expressivity_incl": "RRESTRCUCINTUNIVRESTREROIF(D)",

end up being:

"expressivity": "ALCROIFD",
"expressivity_incl": "ALCROIFD",

I have been too long out of this world, @bparsia what do you think? Does it make sense? For example the D is not in brackets as it often is.

bparsia commented 2 weeks ago

This is definitely not a canonical name (I'd guess it really wants to be SROIF(D) though once you have R people tend not to fuss about the number restrictions). I went to compare it with Protege (5.6 or so) but the metrics view doesn't show expressivity anymore :) So maybe this bitrotted?

I think this is an improvement in that it's not broken and would be useful enough :) The other one is just a bug.