search

cubefs / cubefs-csi

CubeFS Container Storage Interface (CSI) plugins.
Apache License 2.0
48 stars 26 forks source link

完全按照文档部署提示：kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name csi.chubaofs.com not found in the list of registered CSI drivers #16

Closed bingoku closed 4 years ago

bingoku commented 4 years ago 
MountVolume.MountDevice failed for volume "pvc-7773ca39-b350-4f77-b771-4aa62119a3f6" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name csi.chubaofs.com not found in the list of registered CSI drivers

image

k8s中部署的cfs CSI

image

我本地挂载

image

手工挂载在主机上是可以访问的。在K8S中出现上述错误，帮忙解决下咯。

chengyu-l commented 4 years ago

你可以把你的Kubernetes 版本写一下。另外，你可以把external-provisioner日志粘贴一下。文档中还未来的急更新，在部署ChubaoFS CSI，需要将你自己的kubeconfig文件的内容，粘贴到csi-kube-config.yaml文件config值。

chengyu-l commented 4 years ago

查看external-provisioner日志的命令：kubectl logs cfs-csi-controller-xxx-xxx -c external-provisioner，你可以把external-provisioner日志粘贴一下。

bingoku commented 4 years ago

您可以把自己的Kubernetes版本写写一下。另外，您可以把外部provisioner日志记录一下。文档中还未来的急更新，在部署ChubaoFS CSI，需要将自己的kubeconfig文件的内容，粘贴到csi-kube -config.yaml文件配置值。

image

版本

[root@master1 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.8", GitCommit:"ec6eb119b81be488b030e849b9e64fda4caaf33c", GitTreeState:"clean", BuildDate:"2020-03-12T21:00:06Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.8", GitCommit:"ec6eb119b81be488b030e849b9e64fda4caaf33c", GitTreeState:"clean", BuildDate:"2020-03-12T20:52:22Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}

external-provisioner 日志

image

external-attacher 日志

image

I0418 05:22:03.438683       1 feature_gate.go:206] feature gates: &{map[]}
I0418 05:22:03.438760       1 csi-provisioner.go:82] Version: v0.4.2-2-gaa27082c
I0418 05:22:03.438768       1 csi-provisioner.go:96] Building kube configs for running in cluster...
I0418 05:22:03.449710       1 controller.go:134] Connecting to /var/lib/kubelet/plugins/csi.chubaofs.com/csi.sock
I0418 05:22:03.449910       1 controller.go:161] Still trying, connection is CONNECTING
I0418 05:22:03.450145       1 controller.go:161] Still trying, connection is TRANSIENT_FAILURE
I0418 05:22:04.450193       1 controller.go:161] Still trying, connection is CONNECTING
I0418 05:22:04.450303       1 controller.go:158] Connected
I0418 05:22:04.450912       1 controller.go:572] Starting provisioner controller csi.chubaofs.com_cfs-csi-controller-786b78f5b5-2mnpd_8a29e33e-8134-11ea-a5ec-8a079ac0598e!
I0418 05:22:04.451169       1 reflector.go:131] Starting reflector *v1.PersistentVolumeClaim (15m0s) from github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:603
I0418 05:22:04.451179       1 reflector.go:131] Starting reflector *v1.StorageClass (15m0s) from github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:609
I0418 05:22:04.451191       1 reflector.go:169] Listing and watching *v1.PersistentVolumeClaim from github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:603
I0418 05:22:04.451191       1 reflector.go:169] Listing and watching *v1.StorageClass from github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:609
I0418 05:22:04.451174       1 reflector.go:131] Starting reflector *v1.PersistentVolume (15m0s) from github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:606
I0418 05:22:04.451371       1 reflector.go:169] Listing and watching *v1.PersistentVolume from github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:606
I0418 05:22:04.551127       1 shared_informer.go:123] caches populated
I0418 05:22:04.551169       1 controller.go:621] Started provisioner controller csi.chubaofs.com_cfs-csi-controller-786b78f5b5-2mnpd_8a29e33e-8134-11ea-a5ec-8a079ac0598e!
I0418 05:29:15.458393       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:606: Watch close - *v1.PersistentVolume total 0 items received
I0418 05:30:14.062592       1 controller.go:1098] delete "pvc-7773ca39-b350-4f77-b771-4aa62119a3f6": started
I0418 05:30:14.067795       1 controller.go:125] GRPC call: /csi.v0.Identity/GetPluginCapabilities
I0418 05:30:14.067817       1 controller.go:126] GRPC request: {}
I0418 05:30:14.079257       1 controller.go:128] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}}]}
I0418 05:30:14.079917       1 controller.go:129] GRPC error: <nil>
I0418 05:30:14.079930       1 controller.go:125] GRPC call: /csi.v0.Controller/ControllerGetCapabilities
I0418 05:30:14.079934       1 controller.go:126] GRPC request: {}
I0418 05:30:14.082510       1 controller.go:128] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}}]}
I0418 05:30:14.083400       1 controller.go:129] GRPC error: <nil>
I0418 05:30:14.083427       1 controller.go:125] GRPC call: /csi.v0.Identity/GetPluginInfo
I0418 05:30:14.083433       1 controller.go:126] GRPC request: {}
I0418 05:30:14.085467       1 controller.go:128] GRPC response: {"name":"csi.chubaofs.com","vendor_version":"1.0.0"}
I0418 05:30:14.087208       1 controller.go:129] GRPC error: <nil>
I0418 05:30:14.096700       1 controller.go:125] GRPC call: /csi.v0.Controller/DeleteVolume
I0418 05:30:14.096732       1 controller.go:126] GRPC request: {"volume_id":"pvc-7773ca39-b350-4f77-b771-4aa62119a3f6"}
I0418 05:30:14.127803       1 controller.go:128] GRPC response: {}
I0418 05:30:14.128419       1 controller.go:129] GRPC error: <nil>
I0418 05:30:14.128433       1 controller.go:1126] delete "pvc-7773ca39-b350-4f77-b771-4aa62119a3f6": volume deleted
I0418 05:30:14.138091       1 controller.go:1136] delete "pvc-7773ca39-b350-4f77-b771-4aa62119a3f6": persistentvolume deleted
I0418 05:30:14.138136       1 controller.go:1138] delete "pvc-7773ca39-b350-4f77-b771-4aa62119a3f6": succeeded
I0418 05:30:23.455931       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:603: Watch close - *v1.PersistentVolumeClaim total 2 items received
I0418 05:31:28.794381       1 controller.go:927] provision "consul/consul-data-consul-0" class "cfs-sc": started
I0418 05:31:28.809909       1 controller.go:125] GRPC call: /csi.v0.Identity/GetPluginCapabilities
I0418 05:31:28.809952       1 controller.go:126] GRPC request: {}
I0418 05:31:28.811196       1 event.go:221] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"consul", Name:"consul-data-consul-0", UID:"88d12777-d7fb-48ed-a0ab-16d41b6612dd", APIVersion:"v1", ResourceVersion:"406370", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "consul/consul-data-consul-0"
I0418 05:31:28.812363       1 controller.go:128] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}}]}
I0418 05:31:28.813036       1 controller.go:129] GRPC error: <nil>
I0418 05:31:28.813050       1 controller.go:125] GRPC call: /csi.v0.Controller/ControllerGetCapabilities
I0418 05:31:28.813054       1 controller.go:126] GRPC request: {}
I0418 05:31:28.814794       1 controller.go:128] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}}]}
I0418 05:31:28.815615       1 controller.go:129] GRPC error: <nil>
I0418 05:31:28.815627       1 controller.go:125] GRPC call: /csi.v0.Identity/GetPluginInfo
I0418 05:31:28.815631       1 controller.go:126] GRPC request: {}
I0418 05:31:28.832341       1 controller.go:128] GRPC response: {"name":"csi.chubaofs.com","vendor_version":"1.0.0"}
I0418 05:31:28.832765       1 controller.go:129] GRPC error: <nil>
I0418 05:31:28.832803       1 controller.go:433] CreateVolumeRequest {Name:pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd CapacityRange:required_bytes:21474836480  VolumeCapabilities:[mount:<> access_mode:<mode:SINGLE_NODE_WRITER > ] Parameters:map[rdonly:false volName:wsscfs logLevel:debug masterAddr:10.10.10.130:17010 owner:wsscfs] ControllerCreateSecrets:map[] VolumeContentSource:<nil> AccessibilityRequirements:<nil> XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I0418 05:31:28.832973       1 controller.go:125] GRPC call: /csi.v0.Controller/CreateVolume
I0418 05:31:28.832979       1 controller.go:126] GRPC request: {"capacity_range":{"required_bytes":21474836480},"name":"pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd","parameters":{"logLevel":"debug","masterAddr":"10.10.10.130:17010","owner":"wsscfs","rdonly":"false","volName":"wsscfs"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I0418 05:31:28.839664       1 controller.go:128] GRPC response: {"volume":{"attributes":{"logLevel":"debug","masterAddr":"10.10.10.130:17010","owner":"wsscfs","rdonly":"false","volName":"wsscfs"},"capacity_bytes":21474836480,"id":"pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd"}}
I0418 05:31:28.840646       1 controller.go:129] GRPC error: <nil>
I0418 05:31:28.840656       1 controller.go:489] create volume rep: {CapacityBytes:21474836480 Id:pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd Attributes:map[masterAddr:10.10.10.130:17010 owner:wsscfs rdonly:false volName:wsscfs logLevel:debug] ContentSource:<nil> AccessibleTopology:[] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I0418 05:31:28.840702       1 controller.go:552] successfully created PV {GCEPersistentDisk:nil AWSElasticBlockStore:nil HostPath:nil Glusterfs:nil NFS:nil RBD:nil ISCSI:nil Cinder:nil CephFS:nil FC:nil Flocker:nil FlexVolume:nil AzureFile:nil VsphereVolume:nil Quobyte:nil AzureDisk:nil PhotonPersistentDisk:nil PortworxVolume:nil ScaleIO:nil Local:nil StorageOS:nil CSI:&CSIPersistentVolumeSource{Driver:csi.chubaofs.com,VolumeHandle:pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd,ReadOnly:false,FSType:ext4,VolumeAttributes:map[string]string{logLevel: debug,masterAddr: 10.10.10.130:17010,owner: wsscfs,rdonly: false,storage.kubernetes.io/csiProvisionerIdentity: 1587187323449-8081-csi.chubaofs.com,volName: wsscfs,},ControllerPublishSecretRef:nil,NodeStageSecretRef:nil,NodePublishSecretRef:nil,}}
I0418 05:31:28.840911       1 controller.go:1027] provision "consul/consul-data-consul-0" class "cfs-sc": volume "pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd" provisioned
I0418 05:31:28.840936       1 controller.go:1041] provision "consul/consul-data-consul-0" class "cfs-sc": trying to save persistentvolume "pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd"
I0418 05:31:28.846956       1 controller.go:1048] provision "consul/consul-data-consul-0" class "cfs-sc": persistentvolume "pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd" saved
I0418 05:31:28.846988       1 controller.go:1089] provision "consul/consul-data-consul-0" class "cfs-sc": succeeded
I0418 05:31:28.847024       1 controller.go:927] provision "consul/consul-data-consul-0" class "cfs-sc": started
I0418 05:31:28.847154       1 event.go:221] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"consul", Name:"consul-data-consul-0", UID:"88d12777-d7fb-48ed-a0ab-16d41b6612dd", APIVersion:"v1", ResourceVersion:"406370", FieldPath:""}): type: 'Normal' reason: 'ProvisioningSucceeded' Successfully provisioned volume pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd
I0418 05:31:28.855276       1 controller.go:936] provision "consul/consul-data-consul-0" class "cfs-sc": persistentvolume "pvc-88d12777-d7fb-48ed-a0ab-16d41b6612dd" already exists, skipping
I0418 05:31:46.455179       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:609: Watch close - *v1.StorageClass total 0 items received
I0418 05:36:22.460449       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:606: Watch close - *v1.PersistentVolume total 6 items received
I0418 05:37:04.454850       1 reflector.go:215] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:603: forcing resync
I0418 05:37:04.457452       1 reflector.go:215] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:606: forcing resync
I0418 05:37:05.456905       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:609: Watch close - *v1.StorageClass total 0 items received
I0418 05:38:49.457302       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:603: Watch close - *v1.PersistentVolumeClaim total 5 items received
I0418 05:42:08.462502       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:606: Watch close - *v1.PersistentVolume total 0 items received
I0418 05:42:34.466000       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:609: Watch close - *v1.StorageClass total 0 items received
I0418 05:45:19.459497       1 reflector.go:357] github.com/kubernetes-incubator/external-storage/lib/controller/controller.go:603: Watch close - *v1.PersistentVolumeClaim total 0 items received
bingoku commented 4 years ago

cfs-csi-controller-

怎么操作？进pod 里面？

chengyu-l commented 4 years ago

目前ChubaoFS CSI支持K8S 1.12 和CSI 0.3， 你现在的环境是K8S 1.16，K8S 1.6支持CSI 1.0，可能存在兼容性问题。建议你可以使用K8S 1.12进行安装。K8S 1.16会在晚些时候进行内部测试和发布

bingoku commented 4 years ago

目前ChubaoFS CSI支持K8S 1.12 和CSI 0.3， 你现在的环境是K8S 1.16，K8S 1.6支持CSI 1.0，可能存在兼容性问题。建议你可以使用K8S 1.12进行安装。K8S 1.16会在晚些时候进行内部测试和发布

那就期待等你们发新版吧。

bingoku commented 4 years ago

目前ChubaoFS CSI支持K8S 1.12 和CSI 0.3， 你现在的环境是K8S 1.16，K8S 1.6支持CSI 1.0，可能存在兼容性问题。建议你可以使用K8S 1.12进行安装。K8S 1.16会在晚些时候进行内部测试和发布

啥时候才支持1.16和1.17哦。

chengyu-l commented 4 years ago

啥时候才支持1.16和1.17哦。

预计6月底前

aland-zhang commented 4 years ago

我也遇到同样问题k8s 1.18.4

MountVolume.MountDevice failed for volume "pvc-6daf1862-4f12-485e-9a32-0f351bf8a8a6" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name csi.chubaofs.com not found in the list of registered CSI drivers
--
chengyu-l commented 4 years ago

@bingoku @aland-zhang kubernetes 1.17版本以后，已经不支持CSI 0.3.0版本。我已经开发了一个支持CSI 1.1.0 版本， 你可以尝试一下，https://github.com/chengyu-l/chubaofs-csi.git Master分支。目前在稳定性测试阶段，即将合入主库。

需要注意的是，假如使用PVC的Pod被调度到一个Node node_1 上，需要通过给node_1打标签（kubectl label node node_1 chubaofs-csi-node= enabled），在node_1上部署ChubaoFS CSI插件，如下图所示： image

如果希望Kubernetes集群中所有Node都支持ChubaoFS CSI插件，可以将csi-node-daemonset.yaml文件中的nodeSelector节点注释注释（或删除）。

aland-zhang commented 4 years ago

https://github.com/chengyu-l/chubaofs-csi.git Master分支 我测试过了，可以使用。 另外，csi 的 容量限制以及卷扩容的接口都实现了么？

chengyu-l commented 4 years ago

https://github.com/chengyu-l/chubaofs-csi.git Master分支 我测试过了，可以使用。 另外，csi 的 容量限制以及卷扩容的接口都实现了么？

ChubaoFS 提供有扩容接口。CSI的扩容实现也会尽快进行实现。