search

ESIPFed / sweet

Official repository for Semantic Web for Earth and Environmental Terminology (SWEET) Ontologies
Other
116 stars 33 forks source link

URI transition and governance #6

Closed fils closed 7 years ago

fils commented 7 years ago

This issue is created to host discussion around this topic raised in other forums regarding the evolution and governance of the SWEET URIs. In particular:

lewismc commented 7 years ago

I recently added a 2.4 directory to the codebase. I would like to discuss this at our next ESIP SemTech telecon. The only difference in the 2.4 directory, is that the URI for SWEET has been changed from

http://sweet.jpl.nasa.gov/2.3/

to

https://raw.githubusercontent.com/ESIPFed/sweet/master/2.4/

... we need to discuss this in more detail and decide upon a suitable URI...

graybeal commented 7 years ago

Would like to introduce 2 separate elements to this discussion: versioning paths (for ontology and tens); and 'path ownership'. I've found the MMI model for the former to be very powerful for multiple user scenarios.

lewismc commented 7 years ago

Great @graybeal I am all ears... you want to make this an item at this weeks SemTech meetup?

lewismc commented 7 years ago

Please, when considering this issue, can people consult and consider the following documentation http://marinemetadata.org/apguides/ontprovidersguide/ontguideconstructinguris I've found it very helpful in better understanding the options, pro's, con's of URI construction.

akthom commented 7 years ago

Ironically enough - the URL above should be https :) https://marinemetadata.org/apguides/ontprovidersguide/ontguideconstructinguris

lewismc commented 7 years ago

LOL @akthom

lewismc commented 7 years ago

@graybeal @akthom @fils @brandonnodnarb @narock @ESIPFed/semtech @carueda @linepouchard any comments on this one?

carueda commented 7 years ago

@lewismc Hi Lewis,

A new URI.. sweet.esipfed.org ? or perhaps a domain not tied to an organization but rather simply managed by ESIP like sweetontology.org or .io

Those domains look great ... but well, the key issue would be long-term persistence ...

I would just like to comment about IRIs and versioning in general:

graybeal commented 7 years ago

Agree with Carlos.

If I imagine SWEET is going through a long-term renaissance, then sweetontology.orghttp://sweetontology.org is a nice permanent home, suitable for any managing organization and archiving repository, even if requiring a bit more resources to keep up.

But if SWEET becomes mostly static, I think the $10 and few hours (minimum!) every year to maintain the site and services would become really irritating to busy volunteers. Then a location hosted by others (e.g., on a ESIPfed or github repository) would be really nice to have.

sweet.esipfed.orghttp://sweet.esipfed.org might save the $10, but not the hours, and will look wrong if another handoff happens in future. Wouldn't go there.

John

From: Carlos Rueda notifications@github.com<mailto:notifications@github.com> Sent: Monday, June 5, 2017 19:38 Subject: Re: [ESIPFed/sweet] URI transition and governance (#6) To: ESIPFed/sweet sweet@noreply.github.com<mailto:sweet@noreply.github.com> Cc: John Graybeal jgraybeal@stanford.edu<mailto:jgraybeal@stanford.edu>, Mention mention@noreply.github.com<mailto:mention@noreply.github.com>

@lewismchttps://github.com/lewismc Hi Lewis,

A new URI.. sweet.esipfed.orghttp://sweet.esipfed.org ? or perhaps a domain not tied to an organization but rather simply managed by ESIP like sweetontology.orghttp://sweetontology.org or .io

Those domains look great ... but well, the key issue would be long-term persistence ...

I would just like to comment about IRIs and versioning in general:

The ontology document containing the current version of an ontology series might be accessible via the IRIhttp://sweetontology.org/phen, as well as via the version-specific IRIhttp://sweetontology.org/phen/2.4. When a new version is created, the ontology document of the previous version should remain accessible viahttp://sweetontology.org/phen/2.4; the ontology document of the new version, called, say,http://sweetontology.org/phen/3.0, should be made accessible via bothhttp://sweetontology.org/phen and http://sweetontology.org/phen/3.0.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ESIPFed/sweet/issues/6#issuecomment-306365351, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABNU0KLGZZFPPl7V2SFyTd-gOnbyOsOBks5sBLu6gaJpZM4M2zs4.

esip commented 7 years ago

I agree with buying domain makes most sense. Staff can maintain domain name, so should not be a big deal. For sweetontology.org we could also by .com,.info and .net for $10 more dollars. Let me know when to buy domain and what name.

E

On Monday, June 5, 2017, John Graybeal notifications@github.com wrote:

Agree with Carlos.

If I imagine SWEET is going through a long-term renaissance, then sweetontology.orghttp://sweetontology.org is a nice permanent home, suitable for any managing organization and archiving repository, even if requiring a bit more resources to keep up.

But if SWEET becomes mostly static, I think the $10 and few hours (minimum!) every year to maintain the site and services would become really irritating to busy volunteers. Then a location hosted by others (e.g., on a ESIPfed or github repository) would be really nice to have.

sweet.esipfed.orghttp://sweet.esipfed.org might save the $10, but not the hours, and will look wrong if another handoff happens in future. Wouldn't go there.

John

From: Carlos Rueda <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');<mailto: notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');>> Sent: Monday, June 5, 2017 19:38 Subject: Re: [ESIPFed/sweet] URI transition and governance (#6) To: ESIPFed/sweet <sweet@noreply.github.com javascript:_e(%7B%7D,'cvml','sweet@noreply.github.com');<mailto: sweet@noreply.github.com javascript:_e(%7B%7D,'cvml','sweet@noreply.github.com');>> Cc: John Graybeal <jgraybeal@stanford.edu javascript:_e(%7B%7D,'cvml','jgraybeal@stanford.edu');<mailto: jgraybeal@stanford.edu javascript:_e(%7B%7D,'cvml','jgraybeal@stanford.edu');>>, Mention < mention@noreply.github.com javascript:_e(%7B%7D,'cvml','mention@noreply.github.com');<mailto: mention@noreply.github.com javascript:_e(%7B%7D,'cvml','mention@noreply.github.com');>>

@lewismchttps://github.com/lewismc Hi Lewis,

A new URI.. sweet.esipfed.orghttp://sweet.esipfed.org ? or perhaps a domain not tied to an organization but rather simply managed by ESIP like sweetontology.orghttp://sweetontology.org or .io

Those domains look great ... but well, the key issue would be long-term persistence ...

I would just like to comment about IRIs and versioning in general:

The ontology document containing the current version of an ontology series might be accessible via the IRIhttp://sweetontology.org/phen, as well as via the version-specific IRIhttp://sweetontology.org/phen/2.4. When a new version is created, the ontology document of the previous version should remain accessible viahttp://sweetontology.org/phen/2.4; the ontology document of the new version, called, say,< http://sweetontology.org/phen/3.0>, should be made accessible via both< http://sweetontology.org/phen> and http://sweetontology.org/phen/3.0.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ ESIPFed/sweet/issues/6#issuecomment-306365351, or mute the thread< https://github.com/notifications/unsubscribe-auth/ABNU0KLGZZFPPl7V2SFyTd- gOnbyOsOBks5sBLu6gaJpZM4M2zs4>.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ESIPFed/sweet/issues/6#issuecomment-306379033, or mute the thread https://github.com/notifications/unsubscribe-auth/ABXGttnmWyGajqJjIwMDlDYg2QTqUKiqks5sBNYCgaJpZM4M2zs4 .

graybeal commented 7 years ago

Per off-line conversation I had with Lewis, I think we should investigate the possibility of tying GitHub maintenance, github.io wiki (served at sweetontology.org), and semantic repository into a single semantic solution. I think it could be pretty elegant.

john

carueda commented 7 years ago

@graybeal @lewismc

Perhaps you are referring to the hosting service provided by Github pages? If by "semantic repository" you mean some backend service (like the ORR backend system, or Bioportal), that would have to be hosted elsewhere as "GitHub Pages is a static site hosting service."

In any case, following Erin's comments, seems like selecting and securing the domain name would be one of the (most immediate) next steps.

lewismc commented 7 years ago

I think working on a proposed architecture diagram for presentation at the next ESIP SemTech meeting would be a good idea. I'm going to try to make this happen.

graybeal commented 7 years ago

yes, I have some ideas about how to stitch those 3 components together.

linepouchard commented 7 years ago

Hi Lewis and all:

Can you please identify the 3 components?

Line

On Tue, Jun 6, 2017 at 2:15 PM, John Graybeal notifications@github.com wrote:

yes, I have some ideas about how to stitch those 3 components together.

On Jun 6, 2017, at 11:12 AM, Carlos Rueda <notifications@github.com< mailto:notifications@github.com>> wrote:

@graybealhttps://github.com/graybeal @lewismchttps://github.com/lewismc

Perhaps you are referring to the hosting service provided by Github pages< https://pages.github.com/>? If by "semantic repository" you mean some backend service (like the ORR backend system, or Bioportal), that would have to be hosted elsewhere as "GitHub Pages is a static site hosting service."https://help.github.com/articles/what-is-github-pages/

In any case, following Erin's comments, seems like selecting and securing the domain name would be one of the (most immediate) next steps.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ ESIPFed/sweet/issues/6#issuecomment-306571417, or mute the thread< https://github.com/notifications/unsubscribe-auth/ABNU0Hj7yoyOGr6-2- QUa77Z1pzTecHxks5sBZaJgaJpZM4M2zs4>.

======================== John Graybeal Technical Program Manager Center for Expanded Data Annotation and Retrieval /+/ NCBO BioPortal Stanford Center for Biomedical Informatics Research 650-736-1632 skype: graybealski

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ESIPFed/sweet/issues/6#issuecomment-306572229, or mute the thread https://github.com/notifications/unsubscribe-auth/ACzCWlhQmmFPO13yyjq0ysvWdNBYx1Hqks5sBZc2gaJpZM4M2zs4 .

--

Line Pouchard

Computational Science Initiative

Brookhaven National Laboratory

Building 725, room 2-189

PO Box 5000

Upton, New York, 11973-5000

ORCID:0000-0002-2120-6521

Phone 631 344-4626

http://www.bnl.gov/compsci

lewismc commented 7 years ago

Hi Line, Yes, what we were discussing was

  1. Github (current ESIPFed sweet repository) for source code management and issue tracking
  2. esipfed.github.io/sweet for documentation and for associating DNS ( sweetontology.org/.io) with redirects to an ontology server
  3. Either semantic portal or COR for persistence and programmatic access to sweet Ontology resources. Note, regarding the issue of transitioning to a new URI, and generally the URI structure and access, the marine metadata documentation resource provides good guidance on this topic.

I think this is ripe for discussion at our next meeting.

carueda commented 7 years ago

Just a meta-comment: When adding comments via email reply, please consider removing the quoted text so the thread is easier to read back in github.

linepouchard commented 7 years ago

Hi Lewis et al: this is very interesting. How do you plan to make the link between github and the repos?

BTW, Semantic Portal documentation is already in a github wiki here: [https://github.com/ESIPFed/Semantic-Portal] accessible from the portal help menu.

@pgopavar Partha, my student, added some python scripts to access the portal APIS.

Line

lewismc commented 7 years ago

Hi @linepouchard

How do you plan to make the link between github and the repos?

I'm personally going to make best effort to work on a basic architecture graphic for our next SemTech meeting. I think that this is something we can hopefully agree on over the summer... I would invite as many reviews as possible. I'll try to post the document prior to the meeting.

To anwer your question however, if we assume that Github is to be used for source code management, each file within Github has a RAW representation. An example is as follows

Both of the above are identical in terms of content, however the access mechanisms are different. The former is for ease of viewing and human interpretation, the latter for machine interpretation, parsing and processing.

If we take the latter of the above manifestations (machine processable), this URI can be used by an Ontology repository to represent yet another manifestation of the ontology which is served with all the functionality provided by the ontology repository platform. In our case, this is either SemanticPortal or else COR. The URI which is then served by the ontology repository is what we need to focus our attention on. That is why we were discussing the content of the MMI documentation on constructing uris.

The above is by no means a proposed solution which has any real backing or consensus yet, however I hope that it will kick off some more conversation and we can move forward.

What do you think?

graybeal commented 7 years ago

@lewismc I think there's a trap in the GitHub solution just above. The inclusion of the version number in the full path https://github.com/ESIPFed/sweet/blob/master/2.4/human.owl suggests to me that it is embedded in the directory organization -- that is, that there is a directory called 2.4, and one called 2.3, and so on.

If I understand that right, it negates a lot of the value of using Git, because you would be manually copying all the files every revision, and it isn't following the built-in Git versioning model.

Whereas if you created a new SWEET release the way I think you just did (when you merged all the SWEET versions into one versioned set of files -- didn't check but I assume you did this), then each branch contains the versioned ontologies. I think you could name the branches, which would give you https://github.com/ESIPFed/sweet/blob/2.4/human.owl. New work would continue in the next branch, and when done could be merged to the master and released with the appropriate tag ("2.5"). Then versions can be easily compared using their tags.

Sorry if I'm not on the right page, but appreciate your help understanding.

graybeal commented 7 years ago

Separately I want to note that while GitHub is used for hosting, the identifier IRIs possibly should not be the GitHub ones. They should definitely be persistent, so I see 4 options:

  1. Host it at a URL that will (by design) never change, like sweetontology.org (per Erin's suggestion above)
  2. Host it at Github at e.g., https://github.com/ESIPFed/sweet/blob/master/human (note the dropped .owl, per Carlos' suggestion)
  3. Host it natively at a repository intended for hosting, like (a) cor.esipfed.org or (b) semantic.esipfed.org (the former is a little more designed for native hosting, the latter for hosting of remote content)
  4. As suggested on another thread, assign persistent identifiers to the ontology — but I'm not sure that solves the semantic web resolution issue.

Only options 2 and 3a embed graceful support for versioning the URLs, and they can be blended to leverage features from both systems (but you'll have decide which system to use as the persistent identifier base). 3b offers slightly different support for submission versioning, but I think not for accessing older term versions -- must check this for both systems. Option 1 could be manipulated to support versioned and unversioned URLs, but would require programming to do so.

Note: @cmungall's comment at https://github.com/ESIPFed/sweet/pull/29#issuecomment-318569136 is relevant to this, and maybe also to the previous comment. He recommends the approach in option 4 above:

I'd encourage the use of logical names for the ontology IRIs (e.g purls) that are not tied to access mechanism (github)

cmungall commented 7 years ago

Note you can still host on github but use PURLs that are independent of github.

For OBO ontologies we run an apache instance on an amazon micro instance that reads from a community maintained set of purl->location mappers and redirects.

See: https://github.com/OBOFoundry/purl.obolibrary.org/

Example: https://github.com/OBOFoundry/purl.obolibrary.org/blob/master/config/envo.yml

$ wget --no-check-certificate http://purl.obolibrary.org/obo/envo.owl
--2017-07-28 16:06:10--  http://purl.obolibrary.org/obo/envo.owl
Resolving purl.obolibrary.org (purl.obolibrary.org)... 52.3.123.63
Connecting to purl.obolibrary.org (purl.obolibrary.org)|52.3.123.63|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/EnvironmentOntology/envo/master/envo.owl [following]
--2017-07-28 16:06:10--  https://raw.githubusercontent.com/EnvironmentOntology/envo/master/envo.owl
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.184.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.184.133|:443... connected.
WARNING: The certificate of ‘raw.githubusercontent.com’ is not trusted.
WARNING: The certificate of ‘raw.githubusercontent.com’ hasn't got a known issuer.
HTTP request sent, awaiting response... 200 OK
Length: 9109220 (8.7M) [text/plain]
Saving to: ‘envo.owl’

I like actually serving the files from github as we can leverage the whole release system

$ wget --no-check-certificate http://purl.obolibrary.org/obo/envo/releases/2017-06-26/envo.owl
--2017-07-28 16:07:11--  http://purl.obolibrary.org/obo/envo/releases/2017-06-26/envo.owl
Resolving purl.obolibrary.org (purl.obolibrary.org)... 52.3.123.63
Connecting to purl.obolibrary.org (purl.obolibrary.org)|52.3.123.63|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/EnvironmentOntology/envo/v2017-06-26/envo.owl [following]
--2017-07-28 16:07:11--  https://raw.githubusercontent.com/EnvironmentOntology/envo/v2017-06-26/envo.owl
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.184.133
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.184.133|:443... connected.
WARNING: The certificate of ‘raw.githubusercontent.com’ is not trusted.
WARNING: The certificate of ‘raw.githubusercontent.com’ hasn't got a known issuer.
HTTP request sent, awaiting response... 200 OK
Length: 9109220 (8.7M) [text/plain]
Saving to: ‘envo.owl.1’

however, some ontologies are too large for github, we generally use S3 here.

You're welcome to use or adapt the infrastructure (or even to use obolibrary purls directly, but of course I appreciate you likely want your own URLs)

I think this is quite similar to https://w3id.org/ (which is another option)

lewismc commented 7 years ago

@graybeal

@lewismc I think there's a trap in the GitHub solution just above. The inclusion of the version number in the full path https://github.com/ESIPFed/sweet/blob/master/2.4/human.owl suggests to me that it is embedded in the directory organization -- that is, that there is a directory called 2.4, and one called 2.3, and so on. If I understand that right, it negates a lot of the value of using Git, because you would be manually copying all the files every revision, and it isn't following the built-in Git versioning model. Whereas if you created a new SWEET release the way I think you just did (when you merged all the SWEET versions into one versioned set of files -- didn't check but I assume you did this), then each branch contains the versioned ontologies. I think you could name the branches, which would give you https://github.com/ESIPFed/sweet/blob/2.4/human.owl. New work would continue in the next branch, and when done could be merged to the master and released with the appropriate tag ("2.5"). Then versions can be easily compared using their tags.

You can check out the way I've just reformed the ontology. We now have actual release which can be found at https://github.com/ESIPFed/sweet/releases From now on the URI should NOT have any versioning in them. It is my intention that we follow the MMI URI construction methodology (or some other one like suggested by @cmungall ) due to the fact that THESE WORK WELL ON LOADS OF OTHER PROJECTS. Essentially, I am very much willing to work out something between those of us who are willing and able to code, which satisfies reasonable URI construction and resolution.

Sorry if I'm not on the right page, but appreciate your help understanding.

Absolutely no problem. Let's hash it out. It's probably important for me to say, I've just spent a full week with @pbuttigieg and have been interacting with @cmungall . IMHO, for the sake of stabilizing the ontology suite, I think we should take action to implement a URI scheme which has already proven itself to be useful in other projects e.g. OBO Foundry, MMI, etc.

lewismc commented 7 years ago

OK folks, @erinmr went ahead and purchased sweetontology.net, .org, .com, .and .info for 10 years. This has been paid for by ESIP. I'm going to go ahead and make an initial pull request to change every URI from existing https://raw.githubusercontent.com/ESIPFed/sweet/master/2.4/... to merely http://sweetontology.net/...