kopeio / etcd-manager

operator for etcd: moved to https://github.com/kubernetes-sigs/etcdadm
Apache License 2.0
164 stars 45 forks source link

How to configure S3 state store region #301

Open loshz opened 4 years ago

loshz commented 4 years ago

I have a successfully running cluster created by kops. However, I recently discovered that the S3 state store bucket was created in the wrong region, so I moved it to a new region using the following steps:

This all works fine, and I'm able to use kops as expected.

However, the problem is with etcd-manager. It's still looking for the S3 bucket in the old region and I can't see where to update this in config. There is no reference to an S3 region in /etc/kubernetes/manifests/etcd-manager.....

BucketRegionError: incorrect region, the bucket is not in 'ap-northeast-1' region at endpoint ''

I tried manually restarting one of the etcd-manager pods to see if it automatically detected the S3 bucket region, but unfortunately that didn't help.

How do I go about updating this?

loshz commented 4 years ago

After looking into the manifest file for this service, I cannot find a reference to an AWs region:

spec:
  containers:
  - command:
    - /bin/sh
    - -c
    - mkfifo /tmp/pipe; (tee -a /var/log/etcd.log < /tmp/pipe & ) ; exec /etcd-manager
      --backup-store=s3://<s3 bucket>/backups/etcd/main
      --client-urls=https://__name__:4001 --cluster-name=etcd --containerized=true
      --dns-suffix=.internal.ap-northeast-1.my-cluster.com --etcd-insecure=false
      --grpc-port=3996 --insecure=false --peer-urls=https://__name__:2380 --quarantine-client-urls=https://__name__:3994
      --v=6 --volume-name-tag=k8s.io/etcd/main --volume-provider=aws --volume-tag=k8s.io/etcd/main
      --volume-tag=k8s.io/role/master=1 --volume-tag=kubernetes.io/cluster/ap-northeast-1.my-cluster.com=owned
      > /tmp/pipe 2>&1

I've also searched every file in the kops S3 state store and cannot find a reference to the ap-northeast-1 region.