Microk8s integration with external Ceph using ceph-rook add-on is requiring more permissions that the ones needed for Ceph CSI
Thus, I am ending up using root admin credentials which turns into a security concern
What Should Happen Instead?
The connection should be able to work just using RBD role permissions rather than root.
The same way that it is described on Ceph CSI documentation
And the same way that works for LXD integration with Ceph just with RBD role permissions to a target pool
Reproduction Steps
# A - RBD only access credentials
sudo microk8s connect-external-ceph \
--ceph-conf /var/snap/microceph/current/conf/ceph.conf \
--keyring /var/snap/microceph/current/conf/ceph.client.management-microk8s.keyring \
--rbd-pool management-microk8s \
--no-rbd-pool-auto-create
Creating namespace rook-ceph-external
namespace/rook-ceph-external created
Configuring Ceph CSI secrets
Traceback (most recent call last):
File "/var/snap/microk8s/common/plugins/connect-external-ceph", line 184, in <module>
main()
File "/snap/microk8s/6529/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/snap/microk8s/6529/usr/lib/python3/dist-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/snap/microk8s/6529/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/snap/microk8s/6529/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/var/snap/microk8s/common/plugins/connect-external-ceph", line 169, in main
import_external_ceph_cluster(ceph_conf, keyring, namespace, rbd_pool)
File "/var/snap/microk8s/common/plugins/connect-external-ceph", line 109, in import_external_ceph_cluster
p = subprocess.run(
File "/snap/microk8s/6529/usr/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/snap/microk8s/6529/usr/bin/python3', PosixPath('/var/snap/microk8s/common/plugins/.rook-create-external-cluster-resources.py'), '--format=bash', '--rbd-data-pool-name=management-microk8s', '--ceph-conf=/var/snap/microceph/current/conf/ceph.conf', '--keyring=/var/snap/microceph/current/conf/ceph.client.management-microk8s.keyring']' returned non-zero exit status 1
# Same permissions required for Ceph CSI
# https://docs.ceph.com/en/latest/rbd/rbd-kubernetes/
cat /var/snap/microceph/current/conf/ceph.client.management-microk8s.keyring
[client.management-microk8s]
key = HERE_IS_MY_KEY
caps mgr = "profile rbd pool=management-microk8s"
caps mon = "profile rbd"
caps osd = "profile rbd pool=management-microk8s"
# B - Ceph Admin / Root credentials (Which I cannot use for production)
sudo microk8s connect-external-ceph \
--ceph-conf /var/snap/microceph/current/conf/ceph.conf \
--keyring /var/snap/microceph/current/conf/ceph.keyring \
--rbd-pool $MICROK8S_CEPH_POOL \
--no-rbd-pool-auto-create
Creating namespace rook-ceph-external
Error from server (AlreadyExists): namespaces "rook-ceph-external" already exists
Configuring Ceph CSI secrets
Successfully configured Ceph CSI secrets
Importing Ceph CSI secrets into MicroK8s
secret/rook-ceph-mon created
configmap/rook-ceph-mon-endpoints created
secret/rook-csi-rbd-node created
secret/rook-csi-rbd-provisioner created
storageclass.storage.k8s.io/ceph-rbd created
Importing external Ceph cluster
=================================================
Successfully imported external Ceph cluster. You can now use the following storageclass
to provision PersistentVolumes using Ceph CSI:
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
ceph-rbd rook-ceph.rbd.csi.ceph.com Delete Immediate true 0s
Summary
Microk8s integration with external Ceph using ceph-rook add-on is requiring more permissions that the ones needed for Ceph CSI
Thus, I am ending up using root admin credentials which turns into a security concern
What Should Happen Instead?
The connection should be able to work just using RBD role permissions rather than root. The same way that it is described on Ceph CSI documentation And the same way that works for LXD integration with Ceph just with RBD role permissions to a target pool
Reproduction Steps