Fix：ceph/rbd/pvc cannot get secrect

TomatoAres commented 4 years ago

k8s-ci-robot commented 4 years ago

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

:memo: Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).

k8s-ci-robot commented 4 years ago

Welcome @TomatoAres!

It looks like this is your first PR to kubernetes-incubator/external-storage 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-incubator/external-storage has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

k8s-ci-robot commented 4 years ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: To complete the pull request process, please assign cofyc You can assign the PR to them by writing /assign @cofyc in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[ceph/rbd/OWNERS](https://github.com/kubernetes-incubator/external-storage/blob/master/ceph/rbd/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

TomatoAres commented 4 years ago

/assign @cofyc

TomatoAres commented 4 years ago

/assign @cofyc

TomatoAres commented 4 years ago

I have alreay register to cla.But CI failed becase of cla.

cofyc commented 4 years ago

it does not need secret permission in cluster role anymore, because it now supports userSecretNamespace parameter https://github.com/kubernetes-incubator/external-storage/pull/552/files

you can create secret in one namespace, e.g. rbd-provisioner, then configure this role for your provisioner namespace. this is for safety. cluster-level secret permission is too wide for a provisioner.

TomatoAres commented 4 years ago

@cofyc I have tried many times according to readme,and I am in RBAC mode ,my namespace is default.But always failed,I just tried it again:

[root@k8s-master01 examples]# kubectl create -f examples/class.yaml
error: the path "examples/class.yaml" does not exist
[root@k8s-master01 examples]# cd ..
[root@k8s-master01 rbd]# kubectl create -f examples/class.yaml
storageclass.storage.k8s.io/rbd created
[root@k8s-master01 rbd]# kubectl create -f examples/claim.yaml
persistentvolumeclaim/claim1 created
[root@k8s-master01 rbd]# kubectl get pvc
NAME     STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
claim1   Pending                                      rbd            5s
[root@k8s-master01 rbd]# kubectl describe pvc claim1 
Name:          claim1
Namespace:     default
StorageClass:  rbd
Status:        Pending
Volume:        
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-provisioner: ceph.com/rbd
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Mounted By:    <none>
Events:
  Type     Reason                Age                From                                                                               Message
  ----     ------                ----               ----                                                                               -------
  Normal   ExternalProvisioning  10s (x3 over 16s)  persistentvolume-controller                                                        waiting for a volume to be created, either by external provisioner "ceph.com/rbd" or manually created by system administrator
  Normal   Provisioning          0s (x2 over 15s)   ceph.com/rbd_rbd-provisioner-98b88f5d6-vh95q_4c8450aa-272e-11ea-a49d-cef16b3a1441  External provisioner is provisioning volume for claim "default/claim1"
  Warning  ProvisioningFailed    0s (x2 over 15s)   ceph.com/rbd_rbd-provisioner-98b88f5d6-vh95q_4c8450aa-272e-11ea-a49d-cef16b3a1441  failed to provision volume with StorageClass "rbd": failed to get admin secret from ["kube-system"/"ceph-admin-secret"]: secrets "ceph-admin-secret" is forbidden: User "system:serviceaccount:default:rbd-provisioner" cannot get resource "secrets" in API group "" in the namespace "kube-system"
[root@k8s-master01 rbd]# kubectl get pod
NAME                              READY   STATUS    RESTARTS   AGE
rbd-provisioner-98b88f5d6-vh95q   1/1     Running   0          2m18s
[root@k8s-master01 rbd]# kubectl get secrets --all
--all-namespaces               --allow-missing-template-keys  
[root@k8s-master01 rbd]# kubectl get secrets --all-namespaces |grep ceph
kube-system       ceph-admin-secret                                kubernetes.io/rbd                     1      6m28s
kube-system       ceph-secret                                      kubernetes.io/rbd                     1      5m35s

After I add the auth of get secrect,it works.

cofyc commented 4 years ago

do you install role/rolebinding in kube-system namespace?

https://github.com/kubernetes-incubator/external-storage/tree/master/ceph/rbd/deploy#install-with-rbac-roles

cd $GOPATH/src/github.com/kubernetes-incubator/external-storage/ceph/rbd/deploy
NAMESPACE=kube-system
sed -r -i "s/namespace: [^ ]+/namespace: $NAMESPACE/g" ./rbac/clusterrolebinding.yaml ./rbac/rolebinding.yaml
kubectl -n $NAMESPACE apply -f ./rbac

This will create this role and given get kube-system/secrets permission for rbd-provisioner.

cofyc commented 4 years ago

with userSecretNamespace parameter, permission in one namespace is required right now.

antonyjohnson-js commented 4 years ago

Hi there,

Am new to this technology so please ignore me if anything am asking stupidly .. am very curious to know how cephrbd provisioner works with csi increamental backup with k8s.. Can anyone educate me ..

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

TomatoAres commented 4 years ago

/close

kubernetes-retired / external-storage

Fix：ceph/rbd/pvc cannot get secrect #1266