invenio-github docs: add module documentation

[alejandromumo]

• [ ] The current draft docs do not contain information on the new architecture of the module. This should be documented.
• [ ] Additionally, we should add REST API reference docs
• [ ] Add documentation on the new remote application, how to import it and configure it.

[alejandromumo]

Invenio-Github

This was copied from the CodiMD above, which is not accessible outside CERN.

TL;DR setup

Find below step-by-step instructions on how to setup invenio-github on RDM and receive webhooks from Github in your local application:

1. Create a Github Oauth App.
2. Setup a webhook mechanism
3. Install invenio-github and invenio-webhook( invenio-rdm-records will depend on invenio-github and everything should be installed from there).
4. Add invenio-github configs

Testing the backend

Assuming you setup the integration, you can test the backend like so:

• Link a github account. This can be done using the UI (Settings -> Linked accounts -> Github (connect))
• Enable one repo. You need the repository ID for this step.
  • You can retrieve the repo id using github API: GET https://api.github.com/repos/{owner}/{repo_name})
  • Enable the repo in invenio: POST https://127.0.0.1:5000/api/user/github/repositories/<REPO_ID>/enable
• You can create a release on Github and it will create a record in your local instance :rocket:

Other endpoints already tested:

• POST /api/user/github/repositories/sync: syncs all repositories. On Zenodo we have a "Sync now..." button that calls this.

REST API

• POST /api/user/github/repositories/<repository_id>/enable
• POST /api/user/github/repositories/<repository_id>/disable
• POST /api/user/github/repositories/sync
• POST /api/user/github

Testing the UI

1. Connect to Github page:

   • Settings -> Linked accounts -> Github (disconnect) -> GitHub(menu) -> Connect

2. Repositories list page

   • Settings -> Github -> see the list
   • "Get started" header -> "Sync now..." button
   • Repositories list -> enable/disable repos -> reload the page -> repos are sorted by enabled and disabled

3. Repository details page 3.1 Repository with no releases

   • Click on repo with no releases -> enable/disable the repo (different page rendered)
   • "Flip the switch" toggle on a disabled repo page is functioning to enable the repo

   3.2 Repository with releases

   • Click on repo with releases or create a new release in Github -> see the list of releases
   • Click on the header of any release -> see opened tab menu: Citation File, Payload, Metadata(Zenodo specific), Errors(in case there are)

How to create a github OAuth app

Invenio-OAuthClient already provides this doc, however find below the steps to set the github app:

• Login to Github
• Go to https://github.com/settings/developers (github developer settings)
• Hit "New OAuth App" button
• Fill the name of the app (can be anything)
• Under "Homepage URL" you must add your instance's homepage (e.g. https://127.0.0.1:5000/)
• Under "Authorization callback URL" you must add the oauth redirection callback. This should be https://127.0.0.1:5000/oauth/authorized/github/

Retrieve the following client settings:

• Client ID
• Client secrets (create a new client secret if you don't have any)

Setup webhooks

This sets up a way for Github to communicate with a local application.

Ngrok

Ngrok is a service that allows the integration of a local web application to remote applications (e.g. Github).

Basically Ngrok creates a tunnel between their publicly available network to you network. Since ngrok's network is reachable by the internet, Github is able to send packages from their network to ngrok. Ngrok then redirects the packages to you local machine using a tunnel.

Steps

• Access https://ngrok.com/ and create an account

• Install ngrok agent:

  # MacOS
  brew install ngrok/ngrok/ngrok

• Setup ngrok [agent]:

  ngrok config add-authtoken TOKEN

• Setup invenio instance

• Start ngrok tunnel locally.

  For this step, we need to provide the port where the instance is being server (5000 by default). Since we will use it for webhook integrations, we can enforce a secret to be exchanged by Github and Ngrok as an extra security layer.

  Note: there are extra security steps (e.g. adding authentication to connect to the endpoint). See ngrok docs for more information.

   ngrok http https://127.0.0.1:5000 --verify-webhook github --verify-webhook-secret GITHUB_SHARED_SECRET

• The public url for your ngrok session is suffixed by ngrok-free.app. You will need this URL for github to deliver your webhooks. Save it for now. This URL will always be available in your terminal window.

NOTE: when you close the ngrok agent and open a new session, it will assign you a new URL (unless you have a paid version). You might have to change the URL config on invenio if you ever restart the ngrok session.

Docs: https://ngrok.com/docs/getting-started/

Github webhooks cli

https://docs.github.com/en/webhooks-and-events/webhooks/receiving-webhooks-with-the-github-cli

Not fully tested for now, but it should be more convenient than ngrok (and safer).

Invenio configs

The following configs must be added to your instance (invenio.cfg):

from invenio_github.oauth.remote_app import github_app
from invenio_oauthclient.contrib import github
from invenio_rdm_records.services.github.release import RDMGithubRelease

# Add NGROK url to allowed hosts

APP_ALLOWED_HOSTS = ['0.0.0.0', 'localhost', '127.0.0.1', <NGROK_URL>]
# E.g. cea8-2001-1458-204-1-00-102-d4b9.ngrok-free.app

OAUTHCLIENT_REMOTE_APPS = dict(
    github=github_app,
)

OAUTHCLIENT_REST_REMOTE_APPS = dict(
    github=github_app,
)

GITHUB_APP_CREDENTIALS = dict(
    consumer_key="<GITHUB_KEY>",
    consumer_secret="<GITHUB_SECRET>",
)

GITHUB_WEBHOOK_RECEIVER_ID = "github"

# TODO this was blocking webhooks for me, might not be needed though.
REST_CSRF_ENABLED = False

# URL to your app (note it must be reachable from Github)
GITHUB_WEBHOOK_RECEIVER_URL = "<NGROK_URL>/api/hooks/receivers/github/events/?access_token={token}"

GITHUB_INSECURE_SSL = True

GITHUB_SHARED_SECRET = "<GITHUB_SHARED_SECRET>"

GITHUB_RELEASE_CLASS = RDMGithubRelease

GITHUB_INTEGRATION_ENABLED = True

• GITHUB_KEY and GITHUB_SECRET are retrieved from the Github OAuth App

• NGROK_URL is retrieved above

• GITHUB_SHARED_SECRET is the same one used to configure the webhook

Module architecture changes

TODO