Create example integration between Noctua and GitHub using TBD "Simple Integrations API"

[kltm]

This ticket should be closed with either a demonstration, or complete abandonment, of the "simple integrations api".

Notes on simple integrations API, cribbed from the original text for an Apollo idea.

This all really only makes sense with external tools with /real/ REST APIs (likely GitHub, Slack, etc.). A mile high implementation looks like:

• config blob containing button type and content render information
• if necessary pull data from URL (using API key if necessary)
• translate data into standard format (where most of the "heavy" lifting is done, TBD)
  • internal standard has items like: notification, button->action via REST, etc.
  • as the created blob could be standardized, components and renderings could be shared across systems
• display translated blobs in interface

So, imagine an example with github as a review tool:

setup:

• keys, passwords, and metadata about the button and link types to be rendered, as well any worker (possibly along the lines of the workbenches), is kept compiled on the filesystem, used or deployed as necessary

github get:

• curator looks at model page
• noctua client sees that it has a "plugin" setup and a github API key
• as the URL is defined in the metadata, noctua client gets the data
• in this case, noctua client gets all tickets for this model
• there is a transformation step to take the JSON returned from GitHub into our standardized internal representation; in our case, 1 ticket = 1 annotation review request
  • ones that are marked undone are turned into actionable blobs to be displayed
• these blobs are rendered in the interface with links to tickets in github

github put:

• noctua client sees that it has a github API key and the integration is "on"
  • noctua client creates a renderable blob with predefined REST API actions embedded
• noctua client renders a "review request" button in the UI on model pages
• curator finishes annotation and clicks the button
• noctua client uses the REST call + API key to create a new ticket for review in github
• the above action creates a re-render for the "get" above

The "get" and "put" above are just two states of the same "widget", that is entirely defined by the logic of the GET request transformation.

[kltm]

While thinking about the best way to actually do this, I decided it would be useful to add the open tickets (up to 100 for now) on the landing page for github items that have a model name in the title. It will be slow until we add the model regexp to the config data (to make scanning a one time thing), but given the number of models we have right now, it's a reasonable demo.

This has been pushed to production for thoughts and review.

[kltm]

In experimenting with the above, I realize that I fell into the trap I was trying to avoid: the landing display should not be the management display--that's the job of GitHub and why I'd be trying to integrate, it is the manager. The only things I want are:

On a model, a button that

• when clicked, creates a new ticket that:
• contains the model id in the title
• has the tag/label request review, or similar
• any other descriptive data in the body (maybe model comments?)
• is not clickable if a ticket as above has been reviewed and closed

The only hiccup here is that the noctua model itself (model annotations), and the external github metadata may not be the same. This could be fixed at the application level by having the application peak inside the widget and update the model accordingly. No more evil than some other things we've done.

[kltm]

The transformations (REST results -> renderable JSON blob) themselves can be language specific sharable libraries. The widgets and the methods to pass configuration and context information to the target URLs are going to be more specific. That said, if using a framework like Vue.js, that can pretty much be shared as a whole as well (embeddable widgets with external configuration). For server-side integrations, that would be more code, but I suspect that a lot of this could be reused across platforms--e.g. a github manager integration for Apollo could largely be reused in Noctua, and vice versa.

I think that the way forward for us here would be using a pattern much like our workbenches; essentially workbenches as embedded buttons and widgets.

[kltm]

A note on this to @cmungall and @dougli1sqrd, as I have mentioned frustration with the GitHub UI API in the past; while undocumented, I discovered that this actually works as one would expect: https://github.com/geneontology/noctua-models/issues/new?title=foo&body=your+body+here So, technically brittle (use of undocumented API, AFAICT), but I'd guess stable and usable.

[dougli1sqrd]

Oh that's actually pretty cool!

[kltm]

It turns out that their REST API pretty much works as expected if you just feed it to the UI form as arguments. Which sounds like it's intentional and unlikely to go away.

[kltm]

Okay, I have a first pass at at GitHub "integration" into Noctua on the graph editor pages, but it falls short of the "integrations" API that I imagined.

Instead of a read YAML that gets mapped to a button with actions, I have this. While that is not as "generic" in some ways, it is obviously a sane framework and a few runtime args away from being reusable generically as a GitHub management integration in any JS app. So less language agnostic genericism and more plug-and-play within JS?

All that said, this is a rather fun little side trip. Naturally, there is more that could be done here with things like labels, "hats", and different working groups, but I think this is actually very useful functionality now that we are funneling people into GitHub.