search

mumoshu / sopsed

Spawning and storage of secure environments powered by sops, inspired from vaulted. Out-of-box support for kubectl, kube-aws, helm, helmfile
9 stars 2 forks source link

kube-aws and kubectl : unable to read, no such file or directory #3

Open Vince-Cercury opened 6 years ago

Vince-Cercury commented 6 years ago

Scenario: I have an existing kube-aws cluster v0.9.8. Inside the folder I have:

.sops.yaml contains (I will replace all KMS ARN in this post with MYKMS for safety, but it is a valid key) `creation_rules:

I'm authenticated to AWS and I can call AWS API.

I run those commands successfully

[AWSKMS] INFO[0000] Decryption succeeded arn="MYKMS" [SOPS] INFO[0000] Data key recovered successfully [AWSKMS] INFO[0001] Decryption succeeded arn="MYKMS" [SOPS] INFO[0001] Data key recovered successfully

Those commands make a backup of all my credentials/-key.pem into credentials/-key.pem.bak as well as kubeconfig It also creates new files:

I execute git commit and push, and include the two new files. My credentials folder is still ignored by git as we do not want any key inside GIT, right? Somewhere else on my laptop, I clone the same repository, same branch. I authenticate to AWS and make sure I can make API calls.

I run the same commands, but it does not work:

sops-vault run kube-aws status running kube-aws using vault: kube-aws [AWSKMS] INFO[0000] Decryption succeeded arn="MYKMS" [SOPS] INFO[0000] Data key recovered successfully [AWSKMS] INFO[0001] Decryption succeeded arn="MYKMS" [SOPS] INFO[0001] Data key recovered successfully error: open credentials/apiserver-key.pem: no such file or directory

`sops-vault run kubectl get nodes running kubectl using vault: kubectl [AWSKMS] INFO[0000] Decryption succeeded arn="MYKMS" [SOPS] INFO[0000] Data key recovered successfully
[AWSKMS] INFO[0001] Decryption succeeded arn="MYKMS" [SOPS] INFO[0001] Data key recovered successfully
Error in configuration:

I thought that sops-vault would help me encrypt all the kube-aws key and necessary kubectl admin key, from a default kube-aws setup, commit the result to git, pass it on to a colleague who just needs to clone, authenticate to AWS and run the same commands. This colleague should be able to run any kube-aws command (update for example) and kubectl

What am I missing here?

mumoshu commented 6 years ago

Hi @VinceMD, thx for your feedback!

I execute git commit and push, and include the two new files. My credentials folder is still ignored by git as we do not want any key inside GIT, right?

Ah, yes! You have to manually:

Vince-Cercury commented 6 years ago

thanks @mumoshu.

So I guess you are saying it's ok to store in GIT the public key. I assume it wasn't since kube-aws has a gitignore for everything in credentials/ folder.

Instead of removing .gitignore, I've changed its content to:

*.bak

It think it will work for kube-aws.

However for kubectl:

sops-vault run kubectl get nodes error: unable to read client-key[...]/credentials/admin-key.pem for MYCLUSTER-admin due to open [...]/credentials/admin-key.pem: no such file or directory error: run: failed running /usr/local/bin/kubectl: exit status 1

admin-key.pem has been encrypted by the sops-vault run kube-aws command and backed up as admin-key.pem.bak. I can see that sops-vault run kubectl command backs up kubeconfig and encrypts it. I'm not understanding how that helps since the admin-key.pem is not stored inside the kubeconfig. What did you have in mind?

mumoshu commented 6 years ago

Hi @VinceMD, you can make kubeconfig self-contained by flattening it. sops-vault can then encrypt the self-contained version of kubeconfig by running sops-vault run kubectl for first time. Hope this helps!