Proposed efficient criteria for evaluating registration requests

[cmungall]

PROPOSAL:

Criteria for registration in OBO. The goal is to formalize what is currently an ad-hoc process, to filter out ontologies that are clearly out of scope for OBO. It is intentionally weaker than full conformance, because (a) we want to allow registration of imperfect ontologies (b) partial conformance is easier to quickly and objectively check.

The checks are designed to be fast and objective to evaluate. In future we may be able to run the dashboard on a new ontology to help (https://github.com/OBOFoundry/OBO-Dashboard/issues/2) but as written these checks are trivial for anyone with basic ontology skills to evaluate quickly.

Note this does not change the 2 week rule. Anyone from the OBO community is free to object to the ontology on the basis of any of these checks. However, if no objections are raised in the 2 week period, then the ontology becomes registered. Once registered an ontology is never unregistered.

We use ISO language here for SHOULD/MUST. Anyone is free to raise an objection based on a SHOULD but this is informative and non-blocking. If a MUST is clearly violated then this is basis for exclusion/delaying registration.

• p1 open:
  • MUST be conformant with this principle
• p2 format:
  • MUST be conformant with this principle
• p3 URIs:
  • MUST be conformant, with the caveat the prefix is not yet registered
• p4 versioning:
  • The ontology MUST have a versionIRI
• p5 scope:
  • IF there is significant conceptual overlap or matching labels THEN there MUST be computable linkages (one of: skos, obo:xref, owl) these SHOULD be reasonably comprehensive AND there SHOULD be a written plan (e.g ticket) for coordinating with overlapping ontologies
• p6 definitions:
  • SHOULD conform to principle
  • MUST include some definitions
• p7 relations
  • SHOULD conform
• p9 users:
  • SHOULD state existing users and assumed users
• p10 collaboration:
  • The submitter SHOULD be committed to collaboration
• p11 locus:
  • MUST comply
• p12 naming:
  • All classes MUST have labels
• p16 maintenance
  • The ontology MUST have an open tracker where anyone from the community can submit
  • The maintainers SHOULD be responsive to issues in this tracker

[jamesaoverton]

Thanks @cmungall. I think this is a good level of conformance to require from new projects, and it's clearly expressed.

[matentzn]

MUST include some definitions

1 is enough? Maybe:

All ontology root terms MUST have definitions (this is important to orchestrate cross ontology alignment) All ontology terms SHOULD have definitions

All classes MUST have labels

Maybe relax a bit;

All non-obsolete classes MUST have labels All obsolete classes SHOULD have labels

[mellybelly]

Hello Chris, all, Do you think we could start having some type of provisional status for ontologies that want to become conformant? For example, I was recently in conversations with EDAM, which would have to change its license, but even if it did that, there is overlap with SWO and other OBO ontologies that has not yet been fully defined in terms of the interoperability, but strategy and a potential overlap report would be feedback that could be given and then evaluated in order to become an official part of the library.

What do you think? Cheers, Melissa

On Thu, Feb 13, 2020 at 4:08 PM Nico Matentzoglu notifications@github.com wrote:

MUST include some definitions

1 is enough? Maybe:

All ontology root terms MUST have definitions (this is important to orchestrate cross ontology alignment) All ontology terms SHOULD have definitions

All classes MUST have labels

Maybe relax a bit;

All non-obsolete classes MUST have labels All obsolete classes SHOULD have labels

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/OBOFoundry/OBOFoundry.github.io/issues/1116?email_source=notifications&email_token=AAGCPXACV5ZYRRGV2J6SINTRCVO5PA5CNFSM4KUHQLIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELVKSFY#issuecomment-585804055, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGCPXBNRXAAHBDFLACUV3TRCVO5PANCNFSM4KUHQLIA .

[matentzn]

EDIT:

Please ignore this comment. Its only purpose is to enable namespace reservations to avoid URI clashes in case people want to use OBO tools but dont get admitted to OBO. (I use IRIS like http://purl.obolibrary.org/MY_0001, but the MY ontology is out of scope for OBO) NOT relevant to the ticket.

I am in particular keen on the assumption that once I have proposed a prefix, it is mine, even if OBO decides I am out of scope.

[jamesaoverton]

@matentzn I don't understand. The prefix has to be approved. I guess you can wander out of scope later, but we don't have policies for kicking people out, just for orphans and obsoletes.

@mellybelly I'm all in favour of making OBO interoperable with more projects, such as EDAM, or bringing them into the fold. In addition to the license and overlap questions, there would also case-by-case challenges with PURL management, some adjustments to the registry system, upper-level alignment, etc. But a longer discussion probably deserves its own tracker item.

[matentzn]

@jamesaoverton sorry my typical rant. I changed the comment; please ignore it.

[nlharris]

Note this does not change the 2 week rule. Anyone from the OBO community is free to object to the ontology on the basis of any of these checks. However, if no objections are raised in the 2 week proposal, then the ontology becomes registered. Once registered an ontology is never unregistered.

("in the 2 week proposal" -- presumably that last word should be "period")

So this prescreening ends after 2 weeks with either:

• Ontology fails to meet one or more requirements; not registered
• No objections noted; ontology is registered and goes in queue for full evaluation Is that correct?

The checks are designed to be fast and objective to evaluate.

This is probably known to the OBOFoundry leaders, but is there a protocol or convention for who does the evaluation? Is this something that's discussed at the biweekly meetings?

[cmungall]

Incorporating @matentzn suggestions:

• All ontology root terms MUST have definitions
• At least 20% of terms in the ontology MUST have definitions (the number is arbitrary but I think reasonable)
• There SHOULD be a plan for defining all terms
• classes SHOULD have defining logical axioms where appropriate

[cmungall]

@mellybelly - EDAM would first change the license (better to do this as early as possible), but the overlap with SWO is non-blocking (in fact SWO imports EDAM see #1040)

[cmungall]

@nlharris - thanks, updated original comment

I propose we retain the 2 week period, and the rule if no one objects, it is registered

I also propose that we are explicit in allowing anyone in the community to raise an objection based on these criteria. Ultimately only obo operations people have the permissions to merge a PR, so they would make the final call.

[balhoff]

At least 20% of terms in the ontology MUST have definitions (the number is arbitrary but I think reasonable)

@cmungall this will be hard for an organismal taxonomy ontology.

[nlharris]

@balhoff is there a % that you think would be reasonable?

[matentzn]

Its probably not a very popular opinion, but I think any kind of coverage as a MUST could be problematic. I think the top-level classes should be defined, because they delineate the scope of the ontology (and show us how they should interoperate with other OBO ontologies), and the rest SHOULD be defined. The arbitrary 20% hurdle is basically a way to say: proof to us that you take this criterion seriously; but in reality, you can simply copy the label as the definition to fulfill this hurdle, or otherwise hack yourself past the 20%; I think the SHOULD conveys our sentiment well enough.

[wdduncan]

When requesting a namespace should users submit a document explaining their conformance with this checklist?

[wdduncan]

Just starting to think about this again. In regard's to @cmungall proposals, most are excellent. Two contentious points (for me) are:

1. definitions
2. scope

We've discussed definitions quite a bit already. If we are going to do this computationally, we will have to decided between either all terms must have definitions or definitions are optional. Trying to implement a computation solution that finds some middle ground just seems really difficult. The anatomy ontologies provide a nice counterexample to requiring that that all terms have definitions, But it makes me wonder: If the developers have enough computational acumen to programmatically create such ontologies, can they also computationally generate definitions too?

As for scope, this requirement seems a bit too much for granting a namespace. I suppose we can compare the submitted ontology's term label to others in the Foundry, but how do computationally determine the semantics of the labels?

Also, should we require obo unique labels?

[wdduncan]

I think it be helpful to propose some straightforward guidelines for rejecting the request. E.g.:

• Namespace already taken (obvious)
• Ontology is incoherent

more?

[cmungall]

@balhoff - good point we can make explicit exceptions. This should go in the principles document itself. arguably chemicals in same category, although there is a broader discussion here

@wdduncan - good points. we should make the obvious explicit. I don't believe we have coherency covered under a principle, put it under format?

[matentzn]

I tried to distill all feedback and @cmungall in a complete checklist proposal here: https://github.com/OBOFoundry/OBOFoundry.github.io/pull/1266

[matentzn]

This should be closed now that the checklist is there, and new tickets should be made for items that are not covered.

https://github.com/OBOFoundry/OBOFoundry.github.io/blob/master/docs/RegistrationChecklist.md

[nlharris]

Are you aware of any items that are not covered?

[matentzn]

All items are covered!

[nlharris]

Great! So is it ok for me to close this? I don't want to overstep.

[matentzn]

I will close it :D Dont worry, it can be reopened..