Finalize GitOps updates

osowski commented 4 years ago

[x] Review and update Webhook Token parameters for microservice repos
- Token has been updated and added to all microservice repos as GITOPS_TOKEN. Can be accessed in the GH Actions workflows via ${secrets.GITOPS_TOKEN}.
[x] Update GitHub Actions CI in microservice repos to check-in new app-deploy.yaml to refarch-kc-gitops
- Checkout https://github.com/ibm-cloud-architecture/refarch-kc-gitops
- Replace /services/*/base/config/app-deploy.yaml with updated app-deploy.yaml from Appsody build.
- Commit and push updates back to refarch-kc-gitops
- Create packaged GitHub Action as needed
[x] Create new eda-integration environment and document the flow for root README.md
- Based off of example-credentials
- However, for now this assumes definition of secrets inside the namespace/project already
[x] Merge SCRAM support into upstream microservices
- PRs for KAFKA_USER and KAFKA_PASSWORD have been merged into upstream microservices successfully and verified in the 2020.05 cluster's eda-integration environment.
[x] Merge SCRAM support updates from upstream microservices
- Update /services/*/overlays/*.yaml as needed
- Comment out necessary components from /env/base/kustomization.yaml
[x] Create new ArgoCD deployments in 2020.08 cluster, targeting new eda-integration environment

osowski commented 4 years ago

Updated original comment above - Token has been updated and added to all microservice repos as GITOPS_TOKEN. Can be accessed in the GH Actions workflows via ${secrets.GITOPS_TOKEN}.

osowski commented 4 years ago

Merged @jesusmah SCRAM updates into services via https://github.com/ibm-cloud-architecture/refarch-kc-gitops/pull/16

osowski commented 4 years ago

Pending environment/eda-integration-2020.08 implementation available via https://github.com/osowski/refarch-kc-gitops/tree/master/environments/eda-integration-2020.08.

Some items of note:

The Services created by the new Appsody and OpenLiberty Applications are named differently now and it does not appear that we can control the name in the AppsodyApplication / OpenLibertyApplication spec, so ReeferItgTests.yaml needs to be updated in refarch-kc to point to the new service URLs.
We probably want to change the Service types to ClusterIPs as we are exposing them via Routes and we don't need to do double exposure via NodePorts as well. This change can be done after the CI/CD/GitOps pipeline is complete to test some of our changes.

osowski commented 4 years ago

Initial eda-integration-2020.08 environment merged in via https://github.com/ibm-cloud-architecture/refarch-kc-gitops/pull/18

djones6 commented 4 years ago

GitHub actions CI updates:

[x] refarch-kc-ms: Done (https://github.com/ibm-cloud-architecture/refarch-kc-ms/pull/64)
[x] refarch-kc-order-ms: https://github.com/ibm-cloud-architecture/refarch-kc-order-ms/pull/80
[x] refarch-kc-container-ms: https://github.com/ibm-cloud-architecture/refarch-kc-container-ms/pull/57
[x] refarch-kc-ui: https://github.com/ibm-cloud-architecture/refarch-kc-ui/pull/88
[x] refarch-reefer-ml: https://github.com/ibm-cloud-architecture/refarch-reefer-ml/pull/75

osowski commented 4 years ago

There appears to be an issue with the version of kustomize that ships with the current ArgoCD versions (included in the operator) and it manifests when trying to use a folder structure like the one we have implemented.

Reference: https://github.com/kubernetes-sigs/kustomize/issues/2538

We can specify custom versions of kustomize Link 1 & Link 2, however I am unsure of how this would get rolled into the ArgoCD deployment we have since it is managed by the ArgoCD operator.

The Repo Server component is where the specific kustomize binaries need to live, but the operator spec does not currently detail the availability to support the modifications needed by the core ArgoCD documentation.

I will investigate more upon my return on Monday, 8/24.

ArgoCD Application YAML:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: refarch-kc
  namespace: eda-integration
spec:
  destination:
    namespace: eda-integration
    server: https://kubernetes.default.svc
  project: eda-integration
  source:
    path: environments/eda-integration-2020.08
    repoURL: https://github.com/ibm-cloud-architecture/refarch-kc-gitops
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

osowski commented 4 years ago

The issue encountered above can be replicated locally by running kustomize build https://github....

at 11:02:43 AM in refarch-kc-gitops on  master [$?] at ☸️  eda-integration/c106-e-us-south-containers-cloud-ibm-com:31499/IAM#osowski@us.ibm.com took 3s
   kustomize build https://github.com/ibm-cloud-architecture/refarch-kc-gitops.git/environments/eda-integration-2020.08
Error: accumulating resources: accumulateFile "accumulating resources from './env': '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/env' must resolve to a file", accumulateDirector: "recursed accumulation of path '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/env': accumulating resources: accumulateFile \"accumulating resources from './overlays': '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/env/overlays' must resolve to a file\", accumulateDirector: \"recursed accumulation of path '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/env/overlays': accumulating resources: accumulateFile \\\"accumulating resources from '../base': '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/env/base' must resolve to a file\\\", accumulateDirector: \\\"recursed accumulation of path '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/env/base': accumulating resources: accumulateFile \\\\\\\"accumulating resources from '../../apps/refarch-kc': '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/apps/refarch-kc' must resolve to a file\\\\\\\", accumulateDirector: \\\\\\\"recursed accumulation of path '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/apps/refarch-kc': accumulating resources: accumulateFile \\\\\\\\\\\\\\\"accumulating resources from './overlays': '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/apps/refarch-kc/overlays' must resolve to a file\\\\\\\\\\\\\\\", accumulateDirector: \\\\\\\\\\\\\\\"recursed accumulation of path '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/apps/refarch-kc/overlays': accumulating resources: accumulateFile \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"accumulating resources from '../base': '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/apps/refarch-kc/base' must resolve to a file\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", accumulateDirector: \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"recursed accumulation of path '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/apps/refarch-kc/base': accumulating resources: accumulateFile \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"accumulating resources from '../../../services/orderqueryms': '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/services/orderqueryms' must resolve to a file\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", accumulateDirector: \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"recursed accumulation of path '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/services/orderqueryms': accumulating resources: accumulateFile \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"accumulating resources from './overlays': '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/services/orderqueryms/overlays' must resolve to a file\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\", accumulateDirector: \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"recursed accumulation of path '/private/var/folders/j0/4f6fx6x101v60n7qf0971kn40000gn/T/kustomize-724346891/environments/eda-integration-2020.08/services/orderqueryms/overlays': no matches for OriginalId openliberty.io_v1beta1_OpenLibertyApplication|~X|order-query-ms; no matches for CurrentId openliberty.io_v1beta1_OpenLibertyApplication|~X|order-query-ms; failed to find unique target for patch openliberty.io_v1beta1_OpenLibertyApplication|order-query-ms\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"\\\\\\\\\\\\\\\"\\\\\\\"\\\"\""

at 11:24:20 AM in refarch-kc-gitops on  master [$?] at ☸️  eda-integration/c106-e-us-south-containers-cloud-ibm-com:31499/IAM#osowski@us.ibm.com took 2s
   kustomize version
{Version:3.8.1 GitCommit:0b359d0ef0272e6545eda0e99aacd63aef99c4d0 BuildDate:2020-07-16T05:15:32+01:00 GoOs:darwin GoArch:amd64}

osowski commented 4 years ago

Surprisingly, a typo (or leftover namespace in one of the app-deploy.yaml files) was throwing a red herring error message. Removing the namespace: shipping from the order-query-ms/base/config/app-deploy.yaml allowed the environment to be deployed with all the default versions of ArgoCD and Kustomize.

Some additional investigation discovered that we're going to need to complete https://github.ibm.com/case/gse-eda/issues/105 to adopt the new naming structure I had originally implemented this environment, as the POSTGRES credentials are a part of the base app-deploy.yaml and not any kustomizations. I will work on that later this week, but that does not affect the completion of this item being successfully deployed!

osowski commented 4 years ago

I consolidated the POSTGRES secrets and redeployed using this process and everything worked great. Verified the environment is up and running via an itg-tests run and everything went smoothly!

Now on to the documentation!

djones6 commented 4 years ago

Awesome!

ibm-cloud-architecture / refarch-kc-gitops

Finalize GitOps updates #17