search

andyzhangx / kubernetes-drivers

This repository contains all Kubernetes volume drivers on Azure
MIT License
10 stars 1 forks source link

Support for keytabs? #7

Closed mlbiam closed 6 years ago

mlbiam commented 6 years ago

Passwords are a no-go for us. Is it possible to support keytabs?

andyzhangx commented 6 years ago

could you provide more conext about keytabs? an example is better, thanks.

mlbiam commented 6 years ago

Customer doesn't want user's passwords stored on disk (if it were breached and the user's password was compromised it could lead to liability) so we're generating keytabs and then generating kerberos tickets using kinit. So the process is

  1. generate keytab
  2. run kinit with the keytab
  3. mount using the user's kerberos ticket instead of their password
andyzhangx commented 6 years ago

the mount process happens on every agent node, so if have already set keytab on that agent node, you could use mountOptions like sec=krb5, below is an example config: https://github.com/andyzhangx/kubernetes-drivers/blob/93aa8d1bde427fd722531253bdf6f41284922717/flexvolume/cifs/nginx-flex-cifs.yaml#L20

andyzhangx commented 6 years ago

@mlbiam so for keytab scenario, you would only want to specify username and set mountOptions as vers=3.0,dir_mode=0777,file_mode=0777,sec=krb5, right? I could change the cifs driver to support only username specified and then you could set other mountOptions in the config. BTW, in my example, the password is not stored on disk, it stores in k8s secret, you could set RBAC to only admin access passwrod.

andyzhangx commented 6 years ago

@mlbiam so for keytab scenario, you would only want to specify username and set mountOptions as vers=3.0,dir_mode=0777,file_mode=0777,sec=krb5, right? I could change the cifs driver to support only username specified and then you could set other mountOptions in the config. BTW, in my example, the password is not stored on disk, it stores in k8s secret, you could set RBAC to only admin access passwrod.

mlbiam commented 6 years ago

Thanks @andyzhangx i haven't tried to test yet (need to upgrade my openshift to 3.9). Were there any special selinux permissions I need to set?

andyzhangx commented 6 years ago

@mlbiam I only tested this cifs driver on Ubuntu, not sure whether it works on redhat, you may try it first manually, it that works, I will add a new option to support specify username only.

mlbiam commented 6 years ago

@andyzhangx some progress! The good news is that the flexmount runs, and creates volume. Here's where we're having an issue and we're not sure if this is an issue with the flexmount or with OpenShift. When the mount runs, its mounted in the container as root:root with strict enforcement, so the volume accessible in the container. It sounded like an SELinux issue known to cifs file systems so we updated init with log '{"status": "Success", "capabilities": {"attach": false,"selinuxRelabel",false}}' but that hasn't fixed it. The other item thats odd is if I run the container as root, i gen an SELinux error which I'm pretty sure i know how to fix but non root ids aren't able to access the mount at all. Any ideas as to the issue?

andyzhangx commented 6 years ago

@mlbiam did you set dir_mode=0777,file_mode=0777 in mountOptions, also you could set gid=x,uid=x in mountOptions too: https://github.com/andyzhangx/kubernetes-drivers/blob/d66a21ef02457d9c817a0a7c2751eb62c01ee216/flexvolume/cifs/nginx-flex-cifs.yaml#L20

For the second issue, have you tried manually mount cifs on node and run a docker container with volume mapping?

mlbiam commented 6 years ago

@andyzhangx yep, issue was I didn't form the JSON properly and I had to run chcon -Rt svirt_sandbox_file_t /var/lib/origin/openshift.local.volumes to get it to work from inside the container. Otherwise its working great! need to think about keytab distribution but otherwise its a great script! I'm going to do some cleanup and do a pr to share updates