Multi Cluster support

[alexeykazakov]

Users can be provisioned to different OS clusters. So, OSIO services should handle it properly.

There are two main scenarios:

1. A user (space owner/collaborator) is working on a space/codebase in the same cluster.
2. A user (space collaborator) is working on a space/codebase which was created by another user and may be in a different cluster.

We will need to provide API which can be used to get the information about user's cluster. This info should be taken into account when working with a space (space owner and space collaborators may be in different clusters).

Initial strategy to manage which cluster users get to is that the registration app will default to the first cluster and we'll manually choose which users goto the other clusters. Once that is working we'll work on figuring out how to automatically do it - most likely will be either random choosing one or choose the one with the least load.

[alexeykazakov]

We need every OSIO team to prepare requirements for their services/parts. What info (provided by some new API) they would need to get to successfully support multi cluster. Any limitations and gotchas?

[alexeykazakov]

Tasks to be done to get Multi cluster support:

• [x] New Cluster(s) available and ready to get new users provisioned.
• [x] OSO/OSIO registration app provisions OSIO users to multiple clusters instead of the only currently available starter-us-east-2 based on some metrics/capacity etc. OSO registration App.
• [x] Information about user's cluster should be available on the OSIO side. As first step the OSIO/OSO registration app can pass this info to the Auth service when user is approved and Auth passes it along to the tenant service. This workflow can change later as part of our registration/provisioning workflow refactoring. OSO registration App, Auth service, Tenant service. https://github.com/fabric8-services/fabric8-auth/issues/233

Steps to be done until we have OSO Proxy in place:

• [x] Move Account Linking from Keycloak to Auth. New REST API to link accounts and to get OSO (and GitHub) tokens. We will be able to get rid of OSO account linking as soon as we implement OSO proxy. Auth service.

OSO Proxy:

• [x] OSO proxy implementation which uses the OSO service accounts and impersonates OSO users.
• [ ] OSO Proxy Metrics
• [ ] OSO Proxy expose redirect to OSO Console

New Proxy service

• [ ] All OSIO services/clients talk to that Proxy instead of accessing OSO directly. OSO Clients on OSIO side:
  • [x] UI (pipelines/applications) - https://github.com/openshiftio/openshift.io/issues/1685
  • [x] Che - https://github.com/openshiftio/openshift.io/issues/1683
  • [x] Tenant service - https://github.com/openshiftio/openshift.io/issues/1687
  • [x] Forge/app-wizard - https://github.com/openshiftio/openshift.io/issues/1684
  • [x] Jenkins (uses service account tokens) - https://github.com/openshiftio/openshift.io/issues/1686
  • [x] Jenkins Proxy/Idler - https://github.com/openshiftio/openshift.io/issues/1245
  • [ ] Deployments https://github.com/openshiftio/openshift.io/issues/2096

[aslakknutsen]

I've noted down these as currently missing for the complete flow between registration, auth and tenant. (they might overlap with some above)

• [x] the OS registration app doesn't send us the cluster info - https://github.com/openshift/online-registration/issues/762
• [x] auth can't receive the cluster info - https://github.com/fabric8-services/fabric8-auth/issues/162
• [x] auth doesn't send the cluster info to tenant service
• [x] tenant can't receive the cluster info
• [x] auth can't store the cluster-wide tokens - https://github.com/fabric8-services/fabric8-auth/issues/179
• [x] tenant service can't get the cluster tokens from auth

@alexeykazakov Could you get @xcoulon up to speed on these and get the basic plubming for the fow up and running?

[alexeykazakov]

Sure @aslakknutsen. And I will update the list above with links to the issues we are currently working on.

[adisakala]

@surajssd

[sbose78]

the flow should change to tenant init on user registration/approval instead of first access

@alexeykazakov Is this something that we need to do on the Auth side after user creation is triggered from online-registration ?

[alexeykazakov]

@sbose78 yes. We should do it as soon as the registration app switches to the new API. I've created an issue for that - https://github.com/fabric8-services/fabric8-auth/issues/226

[joshuawilson]

@maxandersen SEVerity is only for bugs.

[arunkumars08]

@alexeykazakov Is the proxy url ready and if so, is there a place where I can go check on how to consume them? Just curious to know.

[sbose78]

No @arunkumars08 , The proxy url is not available yet for consumption. @aslakknutsen is working on it.

[arunkumars08]

@sbose78 Thanks Shoubik for the update. Will follow up with @aslakknutsen

[fche]

For OSO tenant monitoring, we will only need https://github.com/fabric8-services/fabric8-tenant/issues/369 - the tenant enumeration API that gives us the cluster+tenant route base URLs.

[joshuawilson]

@alexeykazakov @aslakknutsen what are we missing to close this?

[sbose78]

"New Cluster(s) available and ready to get new users provisioned."

The above piece is not done - new cluster is available but not ready to provision users because of the performance issues being tracked by @jfchevrette .