Deploy GB feature branch in new cluster

[veidenberg]

Description

Adds review job in GitLab CI/CD pipeline in order to deploy Genome Browser non-default backend (feature branch) to review app namespace in the new dev cluster.

Related JIRA Issue(s)

ENSWBSITES-2460

Review App URL(s)

http://deploy-review-nkhdev.review.ensembl.org http://another-branch.review.ensembl.org

Example(s)

Example tests:

• Check CI/CD pipeline logs in GitLab
• Check the resources deployed in review namespace: [ens_adm02@wp-np3-32 ~]$ nkhdev -n deploy-review-nkhdev get deploy,svc,ing,cm,secret
• Check genome browser in the review app

You can test deploying your own GB review deployments by creating a new branch from deploy-review-nkhdev. Before branching GB, do the same for ensembl-client (create a branch from deploy-review-nkhdev) - ensembl-client will setup the review namespace and image pull secrets.

Dependencies

Two other repos have updates that this PR depends on:

• The review setup job uses k8s123-migration-gb branch from ensembl-k8s-manifests repo. See the MR for the changes.
• The k8s review namespace is created by ci/cd pipeline triggered by corresponding feature branch in ensembl-client. See the PR for details.

Actions:

1. Approve & merge the MR/PR above.
2. Revert the branchname in GB review job to k8s123-migration

[kamaldodiya]

Setup is still failing https://gitlab.ebi.ac.uk/ensembl-web/ensembl-dauphin-style-compiler/-/jobs/1606360

[veidenberg]

Setup is still failing https://gitlab.ebi.ac.uk/ensembl-web/ensembl-dauphin-style-compiler/-/jobs/1606360

That is expected and the "did not find namespace" error is there on purpose: you need to create the review namespace with ensembl-client branch before branching any backend (in this case gb). That's because ensembl-client sets up pull-secret (otherwise I would have to do this in every backend pipeline - see my comments above). PR description updated.

[kamaldodiya]

kubectl get ns | grep -q "^${CI_COMMIT_REF_SLUG}" || kubectl create ns ${CI_COMMIT_REF_SLUG} 

This will take care of the namespace creation across the repos so don't need to have depenedency on some other repo to create ns. I think this is much cleaner approach than || true

[veidenberg]

kubectl get ns | grep -q "^${CI_COMMIT_REF_SLUG}" || kubectl create ns ${CI_COMMIT_REF_SLUG} 

This will take care of the namespace creation across the repos so don't need to have depenedency on some other repo to create ns. I think this is much cleaner approach than || true

I'd prefer not to create review ns in backend, because we also need to add image-pull-secrets and the corresponding gitlab variables (see my comment above). And we would need gitlab-client branching anyway for the review app to function.