search

OBOFoundry / OBOFoundry.github.io

Metadata and website for the Open Bio Ontologies Foundry Ontology Registry
http://obofoundry.org
Other
165 stars 203 forks source link

Principle #5 scope - automated validation #1015

Open beckyjackson opened 5 years ago

beckyjackson commented 5 years ago

FP 5 - Scope

Automated checks:

  1. Does the ontology have a scope (domain)?
  2. Is it orthogonal to other OBO Foundry ontologies?

Mechanism:

Some ontologies use the domain property in the YAML. If the value is missing from the YAML, this check fails.

Checking orthogonality will be a bit more difficult. Some of the domains are not very specific (e.g., VO has domain: health). We can provide an info message for now if the domain overlaps with another ontology's domain. Unless we have a CV for domain, though, it will be hard to implement.

A scope/domain annotation property for the ontology header would be good to have as well. Possible in IAO?

nataled commented 5 years ago

From the EWG discussion on this:

Check to see if term labels reproduce those found in other ontologies without using original IDs. BioPortal mapping does this. Main idea here is to determine if the scope is too wide. This would be a quick first pass to find potential issues, and will need supplemental manual checking. Do the terms stay within the stated scope of the ontology?

Related issue: #547 (wherein links to ontologies 'domain' fail)

beckyjackson commented 5 years ago

This would be an interesting comparison @nataled - do you think this would be appropriate as part of the dashboard, or perhaps through another report? We've got a lot of information in the dashboard already, and I wouldn't want anything to get lost. I'd personally rather see this as a different report, but I could be convinced otherwise.

nataled commented 5 years ago

I think we'd need to figure out how to do this by hand, just to get an idea of how it should (or if it could) work. To my knowledge we have yet to try it for an ontology review. The Bioportal mapping will only show external ontologies with the same label. We were thinking that if the ontology being reviewed had minted terms that are also found in other ontologies, there could be at least an overlap of terms, and possibly a too-broad scope.

I'm not yet familiar with the ins and outs of ROBOT to be able to answer the question of dashboard vs other report. Perhaps others will have an opinion. Personally, I favor approaches that make things clear over those that capture everything in one place, if those are the choices.

jamesaoverton commented 5 years ago

@nataled This is a good idea. ROBOT only works with one or a few ontologies at a time, and to check this properly we'd have to have all the OBO terms in some sort of database or triplestore. We should do that eventually, and it will be useful for lots of other things. I haven't worked with Bioportal mapping, but I doubt that they'd appreciate us requesting mappings for tens-of-thousands of terms at a time.

nataled commented 5 years ago

If I'm not mistaken, BioPortal just does it. I looked into it once upon a time (when looking at a new ontology), and did find it helpful. Not sure how automated it could be as presented. Here is the relevant mappings for OBI: http://bioportal.bioontology.org/ontologies/OBI/?p=mappings. As you will see, it is done on an ontology-by-ontology basis.

dosumis commented 4 years ago

Current implementation should be modified to take taxon into account (as specified in ontology metadata). It is currently incorrectly flagging ontologies as having overlapping scope where they apply to non-overlapping taxons.

matentzn commented 4 years ago

Reading this: Registry ontologies should cover non-overlapping domains from here. I thought the whole point of "domain" was to group related ontologies; all disease, or phenotype ontologies.. Now, from the "i want my ontology to be totally green on the dashboard"-perspective, it becomes more of an exercise in "changing the metadata so that it sounds like the scope is unique", or am I mistaken? Is their any chance to re-evaluate the purpose of the domain: or has this ship sailed?

jamesaoverton commented 4 years ago

You're right @matentzn. I underestimated the power of the checkmark.

At this point, I think it's valuable to have a small number of broad categories such as "disease", "phenotype", and "experiment" that let us group ontologies. If a blue "i" discourages this, then we should revise the test.

Longer term, we're hoping that the top-level term(s) of each ontology will cleanly point to OBO Core. Then we'll have a hierarchy that we can use to distinguish scopes.

Medium term, we could use the taxon hierarchy to distinguish scopes for many ontologies.

@beckyjackson: Please drop the orthogonality test until we can do better.

matentzn commented 4 years ago

Longer term, we're hoping that the top-level term(s) of each ontology will cleanly point to OBO Core. Then we'll have a hierarchy that we can use to distinguish scopes.

Perfect!

nataled commented 4 years ago

Please note that orthogonality is NOT part of this principle. The automated check should have two components: (1) A statement of scope for the ontology; and (2) Content that reflects the scope.

I think it is very likely that OBO Core will help with this, in that ontologies should (ideally) have one top level node in the core for each domain mentioned in the scope.

dosumis commented 4 years ago

it becomes more of an exercise in "changing the metadata so that it sounds like the scope is unique".

Or to reflect the true specificity of the scope when overly broad terms were used previously. Agree field needs to be controlled though.

For anatomy ontologies - while species independent (or at least taxonomically very broad) anatomy ontologies are useful - I don't believe it is desirable or even possible to have a single ontology in the anatomy domain (I don't think anyone's arguing this at this point). Arguably the same is true for phenotype ontologies. Given this we need a better specification of what orthogonality is. So, where taxoomically broad ontologies exist, it should be understood that taxonomically more specific ontologies don't necessarily contravene the orthogonality principle. Ultimately I'd like to see additional tests for how well integrated any OBO ontology with encopmassing ontologies (e.g. existnace of logical mapping from taxon specific AOs to Uberon and CL) ans with other foundry ontologies more generally (imports & shared patterns)

This might be a specific example of a more general problem. There are many other examples of an ontology doing a good job of modelling a specific part of a domain that could in principle, if not in practice, be covered by an existing, broad ontology (FBbi & OBI; ChEBI and PRO). A test for mapping up from specific to broad might be useful here too.

Medium term, we could use the taxon hierarchy to distinguish scopes for many ontologies.

+1

beckyjackson commented 4 years ago

Please drop the orthogonality test until we can do better.

Done

allysonlister commented 3 years ago

Good evening! Apologies if this is the wrong issue to comment in (please let me know where I could move it if so), but a few months ago I added a domain value to the markdown for http://www.obofoundry.org/ontology/swo.html but the dashboard report at http://obo-dashboard-test.ontodev.com/swo/dashboard.html still shows the domain section as invalid. Is there something I'm missing?

Thanks very much!

matentzn commented 3 years ago

Oh, this is an old dashboard!

Check this one: http://dashboard.obofoundry.org/dashboard/swo/dashboard.html

@jamesaoverton you should probably retire the http://obo-dashboard-test.ontodev.com server

allysonlister commented 3 years ago

Thank you!