Implement handshake with bioportal

[cmungall]

We would like to implement better interoperation with ncbo - ideally they would pull our metadata

Did someone volunteer for reaching out on the obo op call? @proccaserra? Bjoern?

See also #28

[cmungall]

cc @graybeal

[graybeal]

I am working this issue with the BioPortal team. We have Janna Hastings (hastings - at - ebi.ac.uk) as the contact point for your team, is that correct?

[cmungall]

Hi John, I'm not sure why you have Janna as the POC. She was indeed part of the OBO team but she isn't working in ontologies anymore. You can use me as the contact, and this ticket for updates. Thanks!

[graybeal]

Sorry Chris, which ontology are we talking about? The ontology we were working with in February was CHEBI (per my email threads), but that contact person is now set to Gareth Owen.

I do show Janna as sole contact of MF and MFOEM, and contact with many others of ENF. Perhaps these need to be changed?

[graybeal]

Regarding this thread, I think the problems in February were resolved for CHEBI. Are there other issues still on the table? If not this can be closed.

[cmungall]

@graybeal, this thread has nothing to do with CHEBI. I'm one of the maintainers of obofoundry.org, which includes as part an ontology registry.

All OBO ontologies such as GO, Uberon, CL, etc use this registry, and don't update in bioportal. The idea is that BioPortal imports ontology metadata for the OBO subset from the OBO registry

[graybeal]

Ah, I like that idea. My original remark, that we were working on it, was mistaken then -- I thought that was the concern in question, sorry.

We would be happy to talk to you about the metadata interoperability, whenever you are available. Perhaps you or someone from your team can contact me offline at my Stanford email (jgraybeal) or phone?

John

On Nov 2, 2016, at 12:55 PM, Chris Mungall notifications@github.com<mailto:notifications@github.com> wrote:

@graybealhttps://github.com/graybeal, this thread has nothing to do with CHEBI. I'm one of the maintainers of obofoundry.orghttp://obofoundry.org, which includes as part an ontology registry.

All OBO ontologies such as GO, Uberon, CL, etc use this registry, and don't update in bioportal. The idea is that BioPortal imports ontology metadata for the OBO subset from the OBO registry

[cmungall]

OK, I'll follow on email

[lschriml]

Chris -- is this a closed issue ?

[jvendetti]

I thought it might be pertinent for me to comment here about some work I recently did in BioPortal to help us stay in sync with the OBO Foundry.

I wrote a script (obofoundry_sync) that runs on a nightly basis and checks to see if there are any OBO Foundry ontologies that are missing from BioPortal. The script queries the contents of your registry/ontologies.jsonld file and compares to what we have in BioPortal.

As part of writing this script, I had to create a mapping between BioPortal's ontology acronyms and OBO Foundry's ID spaces (see this line). One enhancement I would like to make is to extract out this mapping and put it in a place that's accessible to both groups so that others could perhaps contribute with regard to maintenance. Any thoughts? Should this be entered as a separate GitHub issue?

[cmungall]

From @graybeal:

The mappings have been moved out of code into a config file:

https://github.com/ncbo/ncbo.github.io/blob/master/oboids_to_bpacronyms.json

They are not sure if this is complete, may be 3-5 missing.

We could have a script that grabs this and injects the acronym into our md files. That way we could provide a direct link to BP from individual pages.

[jvendetti]

@cmungall - as of today, BioPortal has entries for all non-obsolete OBO Foundry / Library ontologies. The JSON file that maps our acronyms to your ID spaces is also up to date.

[cmungall]

So what will be easier - we use this to make canonical bioportal links, or bp uses this to redirect our links, e.g. purl.bioont.org/ontologies/FBbt?scheme=OBO ?

On Thu, Sep 5, 2019 at 10:51 AM Jennifer Vendetti notifications@github.com wrote:

@cmungall https://github.com/cmungall - as of today, BioPortal has entries for all non-obsolete OBO Foundry / Library ontologies. The JSON file https://github.com/ncbo/ncbo.github.io/blob/master/oboids_to_bpacronyms.json that maps our acronyms to your ID spaces is also up to date.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OBOFoundry/OBOFoundry.github.io/issues/80?email_source=notifications&email_token=AAAMMOK7BOPEO5MFKGWAHLLQIFBIVA5CNFSM4BN6HZ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6ADOLI#issuecomment-528496429, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAMMOJVTHBP4ZTYN24N6NLQIFBIVANCNFSM4BN6HZ2A .

[nlharris]

Is @cmungall the only person who can address this?

[graybeal]

I just saw the comment of @cmungall above, missed or lost it earlier.

So what will be easier - we use this to make canonical bioportal links, or bp uses this to redirect our links, e.g. purl.bioont.org/ontologies/FBbt?scheme=OBO ?

I'm not sure I understand your comment in this context, a few more lines of pseudocode might help. (I like that an OBO Foundry listing (or page) for an ontology could point to the same ontology in BioPortal (or AgroPortal, or other locations). But I'm not sure in what context 'bp uses this to redirect our links, e.g. purl.bioont.org/ontologies/FBbt?scheme=OBO'.)

A few general observations then.

BioPortal collaborators are actively figuring out how we can create and use more thorough metadata in BioPortal (and all its OntoPortal relatives). AgroPortal is already doing so as you know, unfortunately we can't just pull that code for scalability reasons. So it will be a while before we will have the final answer.

Once we do, I would expect that if BioPortal (or an OntoPortal deployment) is pulling an ontology from OBO Foundry, we would want to pull the metadata also. Similarly, if there is any content you pull from BioPortal or similar resources, you'd want to pull the metadata annotations. (Our API will be handy for that.)

If we really want synchronous metadata, then each system would want to 'merge' the other system's metadata about an ontology (since OntoPortal systems may have curated additional metadata to OBO Foundry's collection). But this is fraught, because the provenance will likely matter and disagreements about what has the correct values may arise.

Maybe the collective metadata model has to manage metadata from each repository separately, and figure out its own rules for merging the answers. But that's another set of merged metadata from each repository!

[nlharris]

can we assign this to someone other than @cmungall?

[matentzn]

Yes, but @cmungall needs to extract the action items here. Cant see any!

[cmungall]

This is really simple

either

1. bioportal provides a URL that resolves OBO prefixes
2. we maintain a bioportal_prefix in our metadata (only need to do this in cases where it differs, as we do for preferred_prefix, for mixed case ID spaces)

[matentzn]

@graybeal Do you have such a URL (that resolves OBO prefixes as described above), or are there any plans to implement?

[graybeal]

I just re-read the whole thread twice and I confess I don't know what "providing a URL to resolve OBO prefixes" means in practice, nor do I see how it relates to the original thread description

We would like to implement better interoperation with ncbo - ideally they would pull our metadata

So first, as to synchronizing metadata: We do pull some of the metadata as a function of bringing the two ontology sets into sync, though mostly we do so manually. To do more than this is in fact complicated, because it depends on updating our metadata infrastructure, which (as discussed) we'd all like to do together to include the same concepts. Are we still on the same page there?

If, on the other hand, this ticket is now about the question Chris raised in https://github.com/OBOFoundry/OBOFoundry.github.io/issues/80#issuecomment-534328002, that seems to me an entirely different question, probably meriting a new ticket. But to be concrete about your question @matentzn, I don't understand how the suggestion (providing a URL to resolve OBO prefixes) would look and act in the system. Can we have a conversation (or detailed spec) that involves pointing to existing metadata, UIs, and/or behaviors in BioPortal? That would be really helpful.

[cmungall]

http://obofoundry.org/ontology/wbphenotype

we have links to OLS etc. We are able to make them as there is a deterministic transform

https://www.ebi.ac.uk/ols/ontologies/wbphenotype

we just add the ontology id onto a base URL

This works MOSTLY for bioportal but we need a system to handle exceptions where either bp has chosen a different id or there is a clash where someone has reserved something on bp and later taken by an obo

in this case this is the URL bioportal uses

https://bioportal.bioontology.org/ontologies/WB-PHENOTYPE

so either we need URLs like

https://bioportal.bioontology.org/ontologies/WBPhenotype?referrer=OBO

where bp uses its own metadata (see Jennifer's message above)

OR

we retain the exceptions in our metadata

[cmungall]

Let's move forward on this

We should extend the OBO metadata schema to include the bp name.

Question: should the list of exceptions be populated from:

• https://github.com/ncbo/ncbo.github.io/blob/master/oboids_to_bpacronyms.json
• OR bioregistry.io

@cthoyt I see you must include mappings for exceptions as https://bioregistry.io/registry/wbphenotype has the right bp URL, do you sync this using the file above?

or maybe we should just pull the browser options directly from bioregistry?

[cthoyt]

No, I wasn't aware this existed.

The Bioregistry has a hub and spoke model for mappings. This means that there are OBO-Bioregistry mappings and BioPortal-Bioregistry mappings. The Bioregistry completely covers OBO Foundry and automatically imports new entries nightly to ensure this. The Bioregistry does not completely cover BioPortal and does not make imports, but it does automatically align BioPortal entries when possible. This alignment can be later manually curated and is also subject to the constraints of having 1-1 mappings and avoid known false positive mappings curated in https://github.com/biopragmatics/bioregistry/blob/main/src/bioregistry/data/mismatch.json.

It would seem that curating these mappings again would be a bad use of effort since the Bioregistry does it already, so importing them would be a good idea (or simply deferring to Bioregistry as an authority of these mappings)

Here are some options for getting these mappings:

1. I just made a new PR with an API endpoint to get exactly this https://github.com/biopragmatics/bioregistry/pull/625. The PR also includes this use case an example that covers 226 OBO-BioPortal mappings, while the linked file https://github.com/ncbo/ncbo.github.io/blob/master/oboids_to_bpacronyms.json appears to only cover 199 mappings. It also lists the 27 OBO Foundry ontologies that are not mapped to BioPortal, mostly (if not completely) full of outdated/inactive/obsolete ontologies.
2. Bioregistry exports all of its cross-registry mappings in SSSOM that can be accessed here https://github.com/biopragmatics/bioregistry/tree/main/exports/sssom

[cmungall]

I think going via 1 sounds best

I had in mind ingesting into the OBO metadata (but disallowing independent curation to avoid SOT confusion), as the site relies on this being in the .md file. However, I haven't kept up with js development on the site and it may be possible to do this dynamically now.

[cthoyt]

Live endpoint is now available at https://bioregistry.io/api/external/mapping/obofoundry/bioportal.

See also https://bioregistry.io/api/metaregistry/obofoundry/FBbt which makes it possible to query bioregistry records based on external registries' prefixes. You can get the mappings directly out of these

[cthoyt]

Hmm now that I'm double checking, the Bioregistry is storing the "id" field from OBO Foundry and not the preferred prefix. Awfully hard to keep these straight since it's not that one is always used

[nlharris]

So is there still work to be done here, or can this issue be closed?

[nlharris]

@graybeal @cthoyt Ok to close this issue?

[graybeal]

I'm not entirely sure. Many things were mentioned and a few are implemented. We have better interoperation in that we do get the basic metadata for OBO Foundry ontologies in order to create a copy in BioPortal. We don't get all the metadata because so far our metadata model is insufficient, but if that is desired I think it is best covered under a separate issue.

So, a qualified yes, if no one objects. Someone can always re-open it later or create a new (narrowly defined) ticket for additional actions.