[providers][P5/M5] Integration with External Providers

NOTE: Document below is written in the style of documentation. TODOs are placed in several places to highlight design choices still to be made.

UCloud providers can expose their services for consumption via the UCloud interface. A provider can provide resources which are supported by UCloud.

Becoming a UCloud Provider

You can register to become a UCloud provider at https://cloud.sdu.dk/TODO. Everyone can become a test provider, production ready providers must be authorized by a UCloud administrator.

It is possible to run a provider alongside a UCloud dev environment. Look here(TODO) for more details about this.

A UCloud provider is defined by the following information:

Metadata: Contains basic information about where to contact the provider. Immutable, cannot be changed later.
Manifest: Contains information about the providers features. Mutable, can be changed on demand.

Metadata

id: myprovider
domain: myprovider.example.com
https: true
env: production

All UCloud requests will be namespaced at /ucloud/${id}. This should allow it to easily live side-by-side with other HTTP endpoints. If HTTPS is in use then the page must use a valid TLS certificate signed by a commonly recognized certificate authority. https must be true if the env is production.

Manifest

users:
  requireExplicitMapping: false # see below

features: # by default, all features are disabled
    compute:
        docker:
            enabled: true
            web: false
            vnc: false
            batch: true
            logs: true
            terminal: false

        virtualMachine:
            enabled: true
            logs: false
            vnc: false
            terminal: false

    storage:
      enabled: true
      shares: false
      projects: true

    blockDevices:
      enabled: true

    ipAddresses:
      enabled: true

The UCloud provider manifest is retrieved from: GET ${PROTOCOL}://${DOMAIN}/ucloud/${id}/manifest. The manifest is cached by UCloud. The provider can force a reload through the user interface or by sending a request to TODO.

UCloud <=> Provider Communication

All communication between UCloud and the provider will be done via HTTP. The documentation for the API will be available as OpenAPI Specification which will be hosted at https://docs.cloud.sdu.dk/api/providers.

During registration the provider will receive a refresh token. This token is opaque and allows the provider to receive an access token. The access token is a short lived token which must be passed in the "Authorization" header as a bearer token. The access token is implemented as a JWT. The contents of the JWT is well documented.

JWT tokens are used in both directions. That is, UCloud will authenticate with the provider using a JWT. The provider will authenticate to UCloud using a JWT.

The JWT is signed using RSA256. A UCloud provider can verify the authenticity using our public certificate available at TODO and verifying that the role is "SERVICE".

Accounting

It is up to the provider how accounting is done and if they wish to push accounting information to UCloud. A provider might, for example, choose to do all of the accounting on their own (including tracking who has access). This would allow a provider to use UCloud just as an interface.

If a provider wishes to use UCloud for accounting then this is possible. UCloud provides an API which allows the provider to charge for a running compute job. The provider may call this API repeatedly to charge the user for their job. UCloud will respond with a payment required if the user's wallet is out of credits. This indicates to the compute provider that the job should be terminated (since they no longer have credit for the job).

User Mapping

End-users authenticate with UCloud using an identity provider. UCloud makes a best-effort attempt at verifying all information about a user. Regardless of how the end-user proves their credentials, their user is mapped into an internal UCloud representation. You can read more about how users authenticate internally with UCloud here.

In communication between UCloud and the provider for requests which involve a user the following information will be included:

{
  "principal": {
    // The UCloud username (unique identifier)
    "username": "DanSebastianThrane#1234",

    // The UCloud role (typically this is USER)
    "role": "USER",

    // The first name(s) of the user
    "firstName": "Dan Sebastian",

    // The last name of the user
    "lastName": "Thrane",

    // A unique numeric identifier (prefer username if possible)
    "uid": 1234,

    // Primary email associated with the user
    "email": "dthrane@imada.sdu.dk",

    // 2FA enabled (UCloud)
    "twoFactorAuthentication": true,

    // Organizational ID (provided by WAYF/IdP)
    "organization": "sdu.dk"
  },

  // Unique identifier for the project
  "project": "string | null"
}

See the provider APIs for more information about where we include this information.

A provider may require an explicit mapping between UCloud users and an internal user. UCloud will always send information about the UCloud user but this can be useful to register a user mapping in the provider's internal system. The workflow is as follows (work-in-progress):

UCloud user initiates connection with provider through the user interface
- When requireExplicitMapping: true this step is required for the user to use any resources from this provider
This triggers a POST /ucloud/${id}/usermap with the principal payload above and a unique identifier (provided by provider)
Provider responds with a 302 redirect and performs internal authentication
Provider saves the mapping between either the username or uid to an internal user
- NOTE: The uid is not guaranteed to be safe for Linux UIDs. For example, uid 0 can be issued to a UCloud user
Provider redirects the user back to POST /api/providers/usermap/${providerId} with the mapping identifier
- The provider should only redirect in case of a successful mapping

SDU-eScience / UCloud