Deploy Cloud Main Connectivity to Internal Gitlab on AAW Prod

[Collinbrown95]

Description

This issue will track the steps that were taken in order to roll out the gitlab cloud main connectivity feature on the AAW production environment.

Dev Changes

• [x] https://github.com/StatCan/aaw-network-policies/commit/8154a3d68c0abe178d2cdc77b17e2f137d2d9289 - Change ports in netpol to be compatible with new ports on cloud main egress gateway

Prod Changes

• [x] https://github.com/StatCan/aaw-network-policies/commit/215f6dd778f7e6f11fb7f1fc9b636f72218ae19a - Add allow-cloud-main-ingress netpol
• [x] https://github.com/StatCan/aaw-network-policies/commit/15c563374406c9e9a6a69d400bc1a6eb48c8a4a7 - add cloud-main-system.yaml to kustomize manifests.
• [x] https://github.com/StatCan/aaw-argocd-manifests/pull/256 - add IstioOperator and Gateway for cloud main egress gateway.
• [x] TODO: we will need to edit https://github.com/StatCan/aaw-argocd-manifests/blob/aaw-dev-cc-00/kube-system/coredns-custom.jsonnet so that there is a coredns entry mapping the internal gitlab IP address to the internal gitlab hostname. For now, I just manually edited the coredns-custom configmap, but we will want to eventually bake this step into the deployment so that it does not require a manual edit.
  • [x] https://gitlab.k8s.cloud.statcan.ca/cloudnative/aaw/daaas-infrastructure/aaw-dev-cc-00/-/merge_requests/179 <-- PR to add extVar for gitlab private IP
  • [x] https://github.com/StatCan/aaw-argocd-manifests/pull/260 <-- PR to add the DNS A record for select cloud main services in coredns
• [x] https://gitlab.k8s.cloud.statcan.ca/cloudnative/aaw/terraform-advanced-analytics-workspaces-infrastructure/-/merge_requests/185 - enable firewall rule collection group on prod firewall
• [x] https://github.com/StatCan/charts/pull/379/files - PR to allow profiles-controller service account to create destinationrule resources.
• [x] https://github.com/StatCan/charts/pull/380 - bump profiles-controller helm chart version
• [x] https://github.com/StatCan/aaw-argocd-manifests/pull/258 - bump profiles-controller helm chart in argocd-manifests
• [x] https://github.com/StatCan/aaw-kubeflow-profiles-controller/pull/89 - fix cloud main controller to clean up istio resources when non-employee user is added to namespace.
• [x] https://github.com/StatCan/aaw-argocd-manifests/pull/259 - implement new image in argocd-manifests

[Collinbrown95]

TODO

• Need to add a netpol to allow egress to gitlab IP from cloud-main-system namespace.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-gitlab
  namespace: cloud-main-system
spec:
  egress:
  - to:
    - ipBlock:
        cidr: XXX.XXX.XXX.XXX/32  # I.e. allow **only** the specified IP address; omitting the real IP address since this is a public repo
  podSelector: {}
  policyTypes:
  - Egress

[Collinbrown95]

Deprecation and Documentation

• [x] Remove test cases from non-employee-exceptions configmap https://github.com/StatCan/aaw-kubeflow-profiles/commit/8f18e2e41ada4555c882e0ea6b3b7508674fa284
• [ ] on the aaw-network-policy, get rid of ports 8080 and 22, these are not used.

Added by souheil:

• [ ] Clean up aaw-profiles-state-controller (I don't think this is active)
• [ ] document modification/extention of exception list and core dns component?