armory / spinnaker-operator

Open Source Spinnaker Kubernetes Operator source code
Apache License 2.0
186 stars 67 forks source link

External Redis config #205

Open j771 opened 3 years ago

j771 commented 3 years ago

Is there a way to setup Spinnaker deployed with the Armory operator to use an external redis? I see there is some documentation here but I tried this and it did not work: https://docs.armory.io/docs/installation/operator-reference/deploy/ Maybe set the configs from the above link and also set the service-settings/redis.yml as stated here: https://spinnaker.io/setup/productionize/caching/externalize-redis/#configure-a-spinnaker-wide-redis

Not sure how to setup the redis.yml for Armory deploy. Would it be something like the following?

spec:
  spinnakerConfig:
    service-settings:
      redis:
        redis.yml : |
          overrideBaseUrl: redis://admin:passw0rd@some.redis.url:6379
          skipLifeCycleManagement: true

    profiles:
      gate:
        redis:
          configuration:
            secure: true

And for deployment environment config?

spec:
  spinnakerConfig:
    config:
      deploymentEnvironment:
        haServices:
           clouddriver:
             enabled: true
             disableClouddriverRoDeck: redis://admin:passw0rd@some.redis.url:6379
             redisMasterEndpoint: redis://admin:passw0rd@some.redis.url:6379
             redisSlaveEndpoint: redis://admin:passw0rd@some.redis.url:6379
             redisSlaveDeckEndpoint: redis://admin:passw0rd@some.redis.url:6379
           echo:
             enabled: true
devorbitus commented 3 years ago

Here is an example of using an external redis https://github.com/armory/spinnaker-kustomize-patches/blob/master/persistence/patch-redis.yml

j771 commented 3 years ago

Thanks, I found that a few days ago and I was able to set it up. I found that I was able to use a k8s secret for the redis url, which is good.

spec:
  spinnakerConfig:
    profiles:
      gate:
        redis:
          configuration:
            secure: true

    service-settings:
      redis:
        overrideBaseUrl: encrypted:k8s!n:<SECRET_NAME>!k:<SECRET_KEY>
        skipLifeCycleManagement: true

Although the Armory operator seems to be stuck in Updating mode for 12 hours and has not deleted the old redis deployed by the operator before switching to the external redis-ha deploy.

I am seeing this in the operator logs:

{"level":"error","ts":1610705811.6199894,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"spinnakerservice-controller","request":"spinnaker-operator/spinnaker","error":"Operation cannot be fulfilled on spinnakerservices.spinnaker.io \"spinnaker\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(zapLogger).Error\n\t/opt/spinnaker-operator/build/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).reconcileHandler\n\t/opt/spinnaker-operator/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:218\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).processNextWorkItem\n\t/opt/spinnaker-operator/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:192\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(Controller).worker\n\t/opt/spinnaker-operator/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:171\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/opt/spinnaker-operator/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/opt/spinnaker-operator/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/opt/spinnaker-operator/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

devorbitus commented 3 years ago

Een regular Halyard based deployments would not remove the previously deployed redis pods.

Note: By setting skipLifeCycleManagement we are telling Halyard to stop deploying/check the status of the Redis instance. If Halyard has already created a Redis instance, you will have to manually delete it. -> https://spinnaker.io/setup/productionize/caching/externalize-redis/

Operator is currently running Halyard under the hood so it would be the same.

j771 commented 3 years ago

Thanks, I thought that maybe the operator would have taken care of that. I will manually delete the old redis, thank you.

j771 commented 3 years ago

I manually deleted the old redis but the spinsvc is still in a Status Updating.

dogonthehorizon commented 1 year ago

@j771 are you still seeing issues with updating the Operator? There were some changes in 1.2.x around how status was being reported that may resolve the behavior you were seeing.

j771 commented 1 year ago

I have updated to the latest version of the operator and when I did the update I did not run into any issues.