AmiGO should support a more full JSON REST API for data retreival

[kltm]

While AmiGO current supports a minimal API (i.e. http://wiki.geneontology.org/index.php/AmiGO_2_Manual:_Linking_To_AmiGO_2), there is a clamour for a more full-bodied API:

Since Planteome's AmiGO2 instance is currently the "warehouse of record" for Planteome ontologies and annotations, it makes the most sense to use as much AmiGO API functionality as we can, perhaps behind a Planteome API wrapper (i.e. www.planteome.org/api/term_search/q=<query>, www.planteome.org/api/term/PO:0001234/json, etc.)

We know we can get JSON for single term data:

    http://dev.planteome.org/amigo/term/GO:0022626/json

...but what about JSON from a (partial) keyword search? This returns HTML: http://browser.planteome.org/amigo/search/ontology?q=plant%20height

It should be noted that while there is interest in moving forward with a new and better AmiGO API based around the JS API, rereading the initial request, the minimal requirements could likely be met with the current perl framework.

[preecej]

Conversation capture:

Justin,

For your short-term aims, given the information, I believe that something could be levered into the current framework relatively easily. (The longer term goals would require more work, with the graph-based stuff outside of AmiGO.)

There is other interest in capturing some very basic GIS elements in AmiGO, so that might be something that would fold into the system.

A minor question about BisQue: is that a matlab based project?

Cheers,

-Seth

On 01/20/2016 06:50 PM, Justin Preece wrote:

That all sounds promising. Thanks for the answers. I look forward to a discussion in a meeting.

I forgot to include these important links for add’l background info. Here is what we provide now (PO data only, a holdover from the Plant Ontology project):

http://planteome.org/web_services

http://planteome.org/services/web_service.php?request_type=term_search&search_value=leaf&inc_synonyms&branch_filter=plant_anatomy&max=20&prioritize_exact_match

http://planteome.org/services/web_service.php?request_type=term_detail&accession_id=PO:0000252

We use these services in more than one external application, and envision greater internal/external use in the Planteome project.

Step 1 is to expand to all ontologies loaded in our golr instance.

Justin Preece *Sr. Faculty Research Assistant, Bioinformatics Jaiswal Lab, Dept. of Botany & Plant Pathology m: 214-288-1510 * _Oregon _State* University *

On Jan 20, 2016, at 6:32 PM, Chris Mungall <cjmungall@lbl.gov mailto:cjmungall@lbl.gov> wrote:

On 20 Jan 2016, at 18:19, Justin Preece wrote:

This is a good conversation. We should add this to the agenda of the next appropriate Planteome Dev call.

Our immediate use-case is to updatethis lightweight, super-fast service (consisting two methods), originally created to serve out the Plant Ontology for desktop and web apps, to handle multiple ontologies in a single AmiGO (GOLR) instance. Then I can use it as the source for client-side ontology annotation for image segments in a Planteome Bisque module, in collaboration with the UCSB/CyVerse image analysis group.

Basically…this. It’s a mock-up hybrid of our AISO desktop tool and the CyVerse Bisque web environment, actual software under development. Note the term data on the right (sourced from a web service) and the annotated segment on the left:

thanks, cool and v helpful

Regarding the API design and implementation, there are the usual questions of aims and audiences:

Do we build big and generic service to enhance the AmiGO API offerings — in a way that also meets Planteome’s more immediate needs — or does Planteome just piggyback on what’s already been built in the name of expediency?

We can make it so these are the same

Related question: Do we worry about a “proper” RESTful implementation, or just set up stable, parameterized URI-based calls as needed?

The new OLS seems to follow-ultra RESTful principles

http://www.ebi.ac.uk/ols/beta/docs/api

http://www.ebi.ac.uk/ols/beta/api/ontologies/po/terms

personally I find all that underscore stuff a bit odd

note also they follow a simple tag-value model, see the synonyms entries

btw here is the main view http://www.ebi.ac.uk/ols/beta/ontologies/po/terms?iri=http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FPO_0009032

Should the Planteome project itself worry about making this service the beginning of a large service backbone for external clients, or just build what we need to progress on our other integrated projects?

I’m of two minds, to be honest: One says “just build the dang methods, hit the data source, return JSON, and get on with it” and the other says, “This is an opportunity to build the foundation of the Planteome public API we specified in our proposal, and it sounds like the GO group would like to do this for AmiGO anyway. Two birds, one stone.”

Let’s have a chat. JE: When would that be?

Justin Preece Sr. Faculty Research Assistant, Bioinformatics Jaiswal Lab, Dept. of Botany & Plant Pathology m: 214-288-1510 Oregon State University

On Jan 20, 2016, at 5:14 PM, Seth Carbon sjcarbon@lbl.gov wrote:

Chris,

Yes, and/or a google doc for J&J to sketch some reqmnts?

I think both are good. Started an issue here: https://github.com/geneontology/amigo/issues/292 .

Cheers,

-Seth

On 20 Jan 2016, at 16:20, Seth Carbon wrote:

Chris,

Yet Another Repo on github to track this?

Why don't we start with an AmiGO ticket and see how it grows from there?

Cheers,

-Seth

[jaiswalp]

See if this helps https://apiary.io/ in building APIs.

[jaiswalp]

Plant Breeding API is using this http://docs.brapi.apiary.io/#

[kltm]

Poking around a little, it seems to be an interesting tool, but I feel a bit squeegee about using non-free tooling in a project. (Oh hi, GitHub!). I'm not entirely sure what one would get over swagger and a mock server however.

[jaiswalp]

I think its free for open source project like ours.

[kltm]

Sorry, I meant "free" as in OSS or libre software, rather than free of charge. (For the former, I'm always happy to pay.)

[jaiswalp]

i get it

[kltm]

Instead of opening a new ticket, let's continue on here. This is from a discussion with @elserj @preecej @hdietze earlier today, and to help fill in @cmungall about what we talked about.

Given some of the technical details about the final use of the JSON API (used for non-JS clients for things like autocomplete, etc.), I no longer believe that we can accomplish these goals extending the current perl API.

Because of this, the API services will initially be a standalone service, probably copied right out of the geneontology/go-numbers-service code (which would likely be folded back in at some point), and separately deployed. While initially just a server wrapper for the JSON API, it would be the seed for an eventual "AmiGO 3", which would be a UI layer over it that was essentially a port of the current AmiGO 2 stack. (This final bit would be in the nebulous future as a deliverable.)

As the geneontology/go-numbers-service already lays out the use of engines and arguments, and I have heard nothing yet that isn't essentially a RESTification of the current bookmark "api" (which we already have), an initial usable effort is not expected to take more than a day, and should be ready during the window of February given by @preecej .

[preecej]

Thanks, Seth. To brass tacks, then...

Immediate use-case: Serve two API methods from planteome.org (via the AmiGO platform) -- ontology term search and ontology term details -- that return ontology data from Planteome's Solr instance for annotating images with ontology terms in the CyVerse Bisque environment. Needs to be fast enough to be integrated with autocomplete boxes and dialogue labels on 3rd-party Bisque client-side interface

Method 1: Term Search Request: URL in the following format, or similar: /amigo/search/term?q= Optional parameters: include_synonyms, ontology_filter, max_results, prioritize_exact_match Response: JSON (0 to many results)

- {
term_search_response: [
{
match: <term name>,
match_type: <term or synonym>,
accession_id: <##:#######>,
ontology: <ontology name>
},
...
]}

Method 2: Term Detail - this already exists! Request: /amigo/term//json Response: JSON (0 to 1 result) as in this example: http://dev.planteome.org/amigo/term/PO:0000050/json

This is a starting point. Feel free to suggest refinements. Thanks!

[cmungall]

What is the expected value of the ontology field?

[preecej]

A distinct namespace representing a particular ontology: gene_ontology, plant_ontology, plant_disease_ontology, traint_ontology, etc.

Justin Preece Sr. Faculty Research Assistant, Bioinformatics Jaiswal Lab, Dept. of Botany & Plant Pathology m: 214-288-1510 Oregon State University

On Jan 29, 2016, at 7:47 PM, Chris Mungall notifications@github.com wrote:

What is the expected value of the ontology field?

— Reply to this email directly or view it on GitHub https://github.com/geneontology/amigo/issues/292#issuecomment-177060116.

[cmungall]

This is currently impossible for amigo as there is no such metadata in any of the existing ontology files.

First you'll want to insert a dc:title annotation on each ontology. Then we can either have a generic ontology owl:annotation query method, where the client passes in the annotationProperty, or a specific method. Either way the annotation needs to be there.

[cooperl09]

Could it use the exiting ontology annotation in the header, rather than the whole namespace? For example: ontology: to ontology: po

Also there is this: default-namespace: plant_trait_ontology default-namespace: plant_ontology

[preecej]

i vote for using default-namespace. this value will be converted to natural language for use by "real-live humans" in a client UI. ;)

[cmungall]

You can't use default-namespace, it's a syntactic directive that exists only in .obo.

I recommend using standard annotation properties

[preecej]

Fine. dc:title annotation on each ontology. Who should make this annotation?

We just need to be able to pull the name of the ontology associated with a term, since I am assuming that our search results will mix ontology terms (PO, GO, TO, EO, etc. all living in one AmiGO data store), and our users would like some clarity in identifying the source of various terms as they annotate images and other data.

Seth: How does this sound to you?

[kltm]

We talked a little about this and it sounds good. We'll chart progress for this part on #305.

[kltm]

@elserj While there is some more work to do (mainly trying to re-enable the numbers services in this fork and getting the payload to be lighter in "autocomplete" situations), you might want to start taking a look at deploying (and otherwise playing with) amigo.js. To invoke, try:

gulp run-amigo-api

TODO:

• [x] try to re-enable the numbers services in this fork (currently numbers services borked)
• [x] get the payload to be lighter in "autocomplete" situations
• [x] remove hardwire to GO GOlr instance

[kltm]

Short of bugs, I think that this has all been implemented for the first pass (with some use cases being met by creating additional search personalities).