OBO Dashboard - Githubissues

jamesaoverton commented 4 years ago

The is a "catchall" issue for progress on the OBO Dashboard. See a test version here:

http://obo-dashboard-test.ontodev.com/

Some have been discussed so far:

Open #1019 [discussed]
Common Format #1018 [discussed]
Identifier Space #1017 [discussed]
Versioning #1016 [discussed]
Scope #1015
Textual Definitions #1010
Relations #981
Documentation #1009
Documented Plurality of Users #1008
Locus of Authority #1007
Naming Conventions #1006
Maintenance #1020

These checks run on Jenkins, where a zip folder containing the results is archived: https://build.obolibrary.io/job/obofoundry/job/OBOFoundry.github.io/job/master/

A full build takes 6 to 7 hours.

Documentation pages have been added for each automated check. They can be accessed from their principle page.

mellybelly commented 4 years ago

I LOVE the new dashboard! Truly fantastic work!

A few questions: Is there a rule to have all properties in RO? See SEPIO report: http://obo-dashboard-test.ontodev.com/reports/principles/fp7-sepio.tsv

Also can we update the license so as to include CC BY 4.0 as allowable?

Finally, what do the final two columns mean exactly?

jamesaoverton commented 4 years ago

Thanks @mellybelly.

For FP7 Relations, our current test is here: http://obofoundry.org/principles/checks/fp_007
- ERROR if the ontology defines a relation (object property) with the same label as an RO relation; for example:
  - SEPIO defines 'expresses' which is also in RO
  - OBI defines 'supplies' which is also in RO
- INFO if the ontology defines relations (object properties) not in RO; I think this is fine, but there is some debate; the specific issue is #981
CC BY 4.0 is an accepted license. The warning for SEPIO is for the wrong license predicate. Many projects have been using an non-existent Dublin Core predicate to indicate the license, see #540
we're revising the table for clarity, but currently
- "Report" means the result of robot report and links to that output as a TSV file
- "Summary" is the worst value of all the previous columns, as a summary of all the checks

mellybelly commented 4 years ago

got it! On it!

nicolevasilevsky commented 4 years ago

I love this, it seems super useful. Great work @beckyjackson and all that were involved.

Minor feature request - could you freeze the top row, so you can still see it when you scroll down?

wdduncan commented 4 years ago

Can we add this to ODK? That way developers can check their ontology locally using a dashboard. I know that can run robot report and output a tsv, but have a visually interface would be a nice enhancement. Or perhaps just putting in a Docker container would be enough.

beckyjackson commented 4 years ago

@nicolevasilevsky - I think that's a great idea. I'll add that in.

@wdduncan - Right now, the dashboard runs over everything at once, but I'm updating it to run over one ontology at a time so that we can automatically run it as things change (e.g., new release, registry metadata is updated...). I'm adding in functionality to click on each ontology ID and see the dashboard details for just that ontology as well (another HTML page).

wdduncan commented 4 years ago

@beckyjackson Thanks! Let me know if I can help.

nicolevasilevsky commented 4 years ago

When I hover over the definitions cell for Mondo, I see the issues (which is very handy!) and it says see ROBOT report for details - but it's not clear to me how to get to that report.

jamesaoverton commented 4 years ago

@nicolevasilevsky Each ontology has its own page with more details. From the main table, click the name of the ontology in the first column. The details page links to the full ROBOT report, at the bottom.

It probably makes sense for us to add a link in the tooltip you’re describing.

Thanks for the feedback!

drseb commented 4 years ago

I like it a lot. Issues I had:

Like @nicolevasilevsky , I also struggled to find the actual report.
I wonder what the sorting of ontologies means? Why not alphabetic? It took me a while to find HPO.
For HPO, it say robot has 3 errors, but in the report it is only one. Why?

matentzn commented 4 years ago

I wanted to jump to explain how the ordering follows the default OBO foundry website ordering, but I have to agree with you seb, it took me a long time to spot the HP in the list. I also think simple alphabetical order would be good. Also I agree with your second comment. The robot report error count seems misleading. There is really just one!

drseb commented 4 years ago

@matentzn and this one error will also be gone with the next release! hooray!

matentzn commented 4 years ago

You are obsessed with having HP being all green :).. I am sure @jamesaoverton et al. will be delighted, as this must have been one of the major incentives of creating the dashboard!

batchelorc commented 4 years ago

I approve of this and will sort the errors.

mcourtot commented 4 years ago

This is fantastic, great work! I'm not quite clear on how to fix 'plurality of users' for DUO. The principle is at http://obofoundry.org/principles/fp-009-users.html, and DUO is being used by many projects, but I don't quite know how to make this explicit in the file. Similarly for scope - are there recommended APs for those? In contrast the overall ROBOT report for DUO is super useful and very clear on what properties are missing, really nice, thanks!

allysonlister commented 4 years ago

Thanks very much for this - like others, I am highly receptive to gamification of such things, and also want to have "green across the board". :-) I'll put in individual comments on each column in their respective issues as required. Appreciate the hard work!

jseager7 commented 4 years ago

I think this is a really informative way to present validation problems. 👍

It would be nice if the report screens could provide brief suggestions for how to fix the problems listed, because some of the messages – such as 'missing usages' under Plurality of Users – are a bit obscure.

Also, the Naming Conventions row mentions the ROBOT report, but it seems to be missing a link to it, which is there in the other rows:

jamesaoverton commented 4 years ago

Thanks for your feedback everyone, it's much appreciated!

@drseb You're right, there's some problem with the error count for HP
@drseb and @matentzn Yes, the sort order matches the OBOFoundry page, which is grouped into tiers (foundry, library, orphan, obsolete), then alphabetical within tiers. We'll reconsider that.
@mcourtot For 'Plurality of Users' we check the duo.md registry entry, not the OWL file. This page has more details on the check. The duo.md file has a tracker but needs a usages list. We'll try to make this more clear.
@jseager7 Thanks, we'll add another link to the ROBOT report.
@mcourtot and @jseager7 For each principle we have the existing web page, then a new web page with details on the check, e.g. Plurality of Users Principle and Plurality of Users Check. On each check page there's a "Fixes" section, matching dashboard errors to suggestions. So I think the main this is to make this more clear.

Thanks again. I'll see if I can quickly fix some of these things.

diatomsRcool commented 4 years ago

Love it! Can we have floating column headers so as I scroll down I know what the columns are?

jamesaoverton commented 4 years ago

@diatomsRcool Thanks! The header should be floating -- that's what I see. What browser/OS are you using?

chris-grove commented 4 years ago

Looks great! I'm still unclear (after reading the documentation page) how to pass the Plurality of Users Principle check, as in what specifically do I need to do add citations etc. to pass this check?

jamesaoverton commented 4 years ago

Thanks @chris-grove, we'll work to make this more clear. Let me know if the following makes sense:

Every OBO ontology has an OBO registry file in the ontology/ directory of this GitHub repository. I think you're specifically interested in ontology/wbphenotype.md. That page has an Edit button with a pencil icon, which takes you to this editing interface where you can update the entry. When you're done, you add a description of your change at the bottom under "Commit changes", select "Create a new branch...", and click "Propose file change". This will make a Pull Request that will be checked by our technical people, then merged.

To pass the specific Plurality of Users check, wbphenotype.md needs a tracker field, like this:

tracker: https://github.com/obophenotype/c-elegans-phenotype-ontology/issues

This process is explained more fully here. But if you haven't done it before, it's a lot to take in at one time.

Note that this fix won't be reflected in the dashboard until we rebuild it. Currently we're rebuilding on an ad hoc basis, but we plan to rebuild on a regular scheduled.

jamesaoverton commented 4 years ago

I haven't fixed anything yet, but I've added a number of specific usability issues to the dashboard repository:

https://github.com/OBOFoundry/OBO-Dashboard/issues

chris-grove commented 4 years ago

@jamesaoverton That's perfect! Thanks! I'll give it a shot now. RE: rebuild/update, sounds good

sbello commented 4 years ago

This looks really great! Like @chris-grove I find the explanations for how to pass some of the requirements unclear. Any chance that the usage check could also look at the number of distinct browsers using the ontology? The ROBOT report of errors is very useful and nicely formatted to make finding the things that need fixing easy.

diatomsRcool commented 4 years ago

@jamesaoverton Chrome

jamesaoverton commented 4 years ago

Thanks @sbello!

I started this issue for tighter links from problems to fixes: https://github.com/OBOFoundry/OBO-Dashboard/issues/12

The current check for Plurality of Users is quite naive: we just look for tracker and usages in the project's registry entry. By "number of distinct browsers" I think you mean looking at server logs? There are technical challenges to that, and I'm not sure that it would address the spirit of the principle. But this is the specific issue for the Plurality of Users automated check, if you'd like to see that discussion and comment on those specific rules: https://github.com/OBOFoundry/OBOFoundry.github.io/issues/1008

sbello commented 4 years ago

@jamesaoverton By distinct browsers I meant looking at the 'browser' section on the OBO foundry page (https://github.com/OBOFoundry/OBOFoundry.github.io/blob/master/ontology/mp.md) The MP entry lists the MGI, RGD, and Monarch browsers and I was wondering if that should/could contribute to the plurality of users check. I'll add a comment to the other ticket as well.

jamesaoverton commented 4 years ago

Thanks @diatomsRcool. That was a bug. I think I fixed it with https://github.com/OBOFoundry/OBO-Dashboard/commit/7764a03c0ff175b1c88a28240e7dd9a2d7a8e3bb

nicolevasilevsky commented 4 years ago

That was a bug. I think I fixed it with OBOFoundry/OBO-Dashboard@7764a03

I was having the same issue in Chrome, and it is fixed now. :)

zhengj2007 commented 4 years ago

Thanks for making the checking automatically. It is very useful. Love it!

It seems Dashboard checked both ontology file and ontology registry file (.md). It would be nice to indicate which file(s) need to be updated for fixing the issues.

Besides, for checking whether the issues fixed in the ontology, I run the robot report command. How can I know whether I fix the issues in .md file? Thanks!

cmungall commented 4 years ago

@sbello - you are right, the addition of custom/MOD browsers is a good proxy for additional users, good idea.

My personal preference is that rather than having the dashboard check this, we have an SOP whereby if we have a custom browser entry this prompts us to manually curate a usage entry, making this more explicit. This is a bit more of a metadata curation burden, but it should be one time, and makes everything more explicit.

It could certainly be documented better!

In any case I don't think this is going to be a problem for any MOD ontology as these usually have tons of data manually annotated using them, I think this is a nice way to demonstrate this!

nataled commented 4 years ago

Hopefully these weren't already mentioned (OBO Dashboard ALPHA 2020-02-23):

1) It's unclear why the mouseover for 'ROBOT report' and 'Summary' give different information. For example, using PR, the report says 5 errors, 3 warnings, and 1 info, while the summary gives only 4 errors.

2) Finding the report itself was not intuitive. Expected to find it linked under ROBOT report.

3) Would be good to have a single resource that explains how to fix certain metadata issues that are not covered by a principle. Even better: a separate column for these, whereby the header shows how the information is collected and fixed. For example, I had to look through 5-6 other ontologies to find one that got missing_ontology_title and missing_ontology_description right (it was DO).

cerivs commented 4 years ago

I nth Nicole's comment about making the header fixed so the column headers stay fixed when you scroll. Some of the "user" documentation seems to overlook other users. Example ZFIN uses CHEBI and PATO among other ontologies we have published this info.

cmungall commented 4 years ago

@cerivs - what would be awesome would an example of how CHEBI is used from the ZFIN web interface, e.g in a query. For the usages field we like to show concrete usages that can be verified in preference to more 'under the hood' usage. If you can provide an example we can add it to the chebi metadata

mah11 commented 4 years ago

General impression: very cool. The checks for duplicate text are especially useful. It'll take us a bit longer to deal with the non-RO relations, but with @matentzn's help we're on it (thanks!).

Specific things:

+1 to the various calls for better linking to the ROBOT report.
The FYPO entry says "missing ontology license", but the fypo.md file has this:

license: url: http://creativecommons.org/licenses/by/3.0/ label: CC-BY

How do I fix it so the dashboard knows it's there (I can also update it to 4.0, whether that's the fix or not)? Or is it a dashboard bug?

cerivs commented 4 years ago

@cmungall http://zfin.org/action/expression/experiment?id=ZDB-EXP-190627-10. CHEBI terms are used with ZECO terms to define what chemicals are used to treat the fish. We have over 1500 different chemical environments using CHEBI terms. Also shown here http://zfin.org/search?q=&category=Phenotype&fq=conditions:%22chemical+treatment+by+environment%3A+EC+1.14.13.39+%28nitric+oxide+synthase%29+inhibitor%22

matentzn commented 4 years ago

@mah11 check new FYPO issue.

cmungall commented 4 years ago

Thanks @cerivs

It would be awesome if your experiment was linked from this page: http://zfin.org/action/ontology/term-detail/CHEBI:63969

But I'm using your experiment page for now: #1136 I also added Rhea

CHEBI folks (@G-Owen) - great if you add more usages to your metadata!

hoganwr commented 4 years ago

I think it looks great! Very helpful.

ANiknejad commented 4 years ago

thank you All, for the hard work!

Anne on behalf of the Bgee team (bgee.org)

zhengj2007 commented 4 years ago

Duplicate definitions of instances are marked as 'ERROR'. (See example, http://obo-dashboard-test.ontodev.com/obi/robot_report.html). During OBI call on Mar 9th, we agree it is an issue. However, it is not big problem. So, we suggest to tag the issue from 'ERROR' to 'WARN'.

nataled commented 4 years ago

I presume you refer to the terms defined as "A supplier of flow cytometry analyzers" correct? (just checking)

The Principles state "Instances, such as organizations or geographical locations, can benefit from definitions although it is understood that definitions for instances are not required."

I agree that the report should downgrade the severity of these.

rvita commented 4 years ago

There is a typo in the error message for the "open" check. It says "...does not match regsitry license" with "registry" misspelled.

martinjramirez commented 4 years ago

Thanks @cmungall ! It looks helpful.

hlapp commented 4 years ago

Satisfying the license requirement is detected how? CDAO is reported as missing a license, but there's a dc:rights assertion in the ontology metadata in the OWL file.

balhoff commented 4 years ago

@hlapp there is a fix in progress for that one: https://github.com/OBOFoundry/OBO-Dashboard/pull/8

ramonawalls commented 4 years ago

I'm not sure if this is the right place to put this, but I don't see a more specific spot. The ROBOT at http://obo-dashboard-test.ontodev.com/bco/robot_report.html says that 6 terms are missing obsolete labels. The two properties were indeed missing the word obsolete, but the three classes listed are not. Why is it picking those up?

Also, I am getting an annotation_white_space warning for the definitions of restricted search taxonomic inventory process [BCO:0000051] | IAO_0000115 and opportunistic search taxonomic inventory process [BCO:0000053], but there are no extra white spaces that I can see in those two. For the other one (material target of observation [BCO:0000044]) there was an extra white space at the end of the comment.

ramonawalls commented 4 years ago

Oh dang! Nevermind. I just realized that I fixed those classes a month ago. Thank goodness for github commit messages. I still have the question about the extra white spaces.

lubianat commented 3 years ago

Nice dashboard! As a small note/feedback, it seems that CC-BY 4.0 licenses are currently being rejected: http://dashboard.obofoundry.org/dashboard/obib/dashboard.html

OBOFoundry / OBOFoundry.github.io

OBO Dashboard #1076