VREPaaS

Development environment

This project is using Django rest framework for the API, and Next.js for the frontend.

Install Anaconda

Install Anaconda from these instructions: https://linuxize.com/post/how-to-install-anaconda-on-ubuntu-20-04/

Close the terminal and start a new one to activate conda.

Dependencies

Create and activate conda environment:

conda env update --file environment.yaml

Install GitGuardian pre-commit hook

pre-commit install
ggshield auth login

Install tilt

Install tilt via conda

conda install -c conda-forge tilt

Install minikube

conda install -c conda-forge minikube

Follow step 3 section 'Linux OS with Network Manager' of the minikube ingress-dns setup guide.

Add secrets

Create tilt/helm-values-secrets.yaml and fill-in the following:

global:
  keycloak:
    url:
    realm:
    client_id:
    client_secret_key:

  argo:
    namespace:
    url:
    token:

Create tilt/helm-n-a-a-vre-secrets.yaml and fill:

hub:
  config:
    GenericOAuthenticator:
      client_id:
      client_secret:
      authorize_url:
      token_url:
      userdata_url:

singleuser:
  extraEnv:
    NAAVRE_API_TOKEN:
    SEARCH_API_ENDPOINT:
    SEARCH_API_TOKEN:
    CELL_GITHUB_TOKEN:

Start Cluster

minikube start
minikube addons enable ingress
minikube addons enable ingress-dns
minikube dashboard  # optional

Run tilt

tilt up

Troubleshooting

Context deadline exceeded when pulling NaaVRE image

If you get Failed to pull image "qcdis/n-a-a-vre-laserfarm:v2.0-beta": rpc error: code = Unknown desc = context deadline exceeded in the continuous-image-puller logs:

Reset the cluster (minikube delete and re-run the startup commands)
Run minikube image load qcdis/n-a-a-vre-laserfarm:v2.0-beta in your terminal
Run tilt

API Admin

To add/remove resources go to: https://paas.minikube.test/vre-api-test/admin/

Admin Credentials

The admin credentials are read from Helm values (e.g. tilt/helm-values-dev.yaml), with the key vreapi.auth.superuser_{username,password}.

Argo Workflows

Generate Token

kubectl apply -f - <<EOF
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: vre-api
  namespace: argo
rules:
  - verbs:
      - get
      - watch
      - patch
      - delete
    apiGroups:
      - ''
    resources:
      - pods
  - verbs:
      - get
      - watch
      - patch
    apiGroups:
      - ''
    resources:
      - pods/log
  - verbs:
      - create
    apiGroups:
      - ''
    resources:
      - pods/exec
  - verbs:
      - list
      - watch
      - create
      - get
      - update
      - delete
    apiGroups:
      - argoproj.io
    resources:
      - workflowtasksets
      - workflowartifactgctasks
      - workflowtemplates
      - workflows
      - cronworkflows
  - verbs:
      - patch
    apiGroups:
      - argoproj.io
    resources:
      - workflowtasksets/status
      - workflowartifactgctasks/status
      - workflows/status
EOF

kubectl create sa vre-api -n argo

kubectl create rolebinding vre-api --role=vre-api --serviceaccount=argo:vre-api -n argo

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  namespace: argo
  name: vre-api.service-account-token
  annotations:
    kubernetes.io/service-account.name: vre-api
type: kubernetes.io/service-account-token
EOF

ARGO_TOKEN="Bearer $(kubectl get secret vre-api.service-account-token -n argo -o=jsonpath='{.data.token}' | base64 --decode)"

echo -n $ARGO_TOKEN | base64 -w 0

Authorization

Token

Create a user in the Django admin panel
Create a token for the user in the Django admin panel
Use the token in the header of the request

resp = requests.get(
    f"{api_endpoint}/api/workflows/",
    headers={
        'Authorization': 'Token '+ naavre_api_token
    }
)

Test Submit Workflow

curl -X POST "http://paas.minikube.test/vre-api-test/api/workflows/submit/" -H "Authorization: Token ${accessToken}" -H "Content-Type: application/json" -d "@vreapis/tests/resources/workflows/submit_workflow_req_body.json"

Releases

If we want to add a new release environment we need to add a new .env.{ENV_NAME} together with a new line in the matrix on the .workflows/make.yaml and .workflows/make-release.yaml

QCDIS / VREPaaS

readme