ceph / ceph-csi

CSI driver for Ceph
Apache License 2.0
1.29k stars 550 forks source link

Implement GetFenceClients to return the ceph clusterID and the client Address #4944

Closed Madhu-1 closed 3 weeks ago

Madhu-1 commented 3 weeks ago

This PR implements a csiaddons RPC calls to return the ceph fsID and the client IP that is used to connect to the ceph cluster from the rbd plugin daemonset pod.

closes: #4913

Madhu-1 commented 3 weeks ago

Test result

sh-5.1# csi-client 
Enter the path for the Unix socket: unix:///csi/csi-addons.sock
Using socket path: unix:///csi/csi-addons.sock
Server response: id:"a815fe8e-eabd-4e87-a6e8-78cebfb67d08"  addresses:{cidr:"10.244.0.1"}
sh-5.1# cat /sys/devices/rbd/0/
block/        client_id     config_info   features      major         name          pool          pool_ns       refresh       snap_id       uevent        
client_addr   cluster_fsid  current_snap  image_id      minor         parent        pool_id       power/        size          subsystem/    
sh-5.1# cat /sys/devices/rbd/0/client_addr 
10.244.0.1:0/353261872
sh-5.1# cat /sys/devices/rbd/0/cluster_fsid 
a815fe8e-eabd-4e87-a6e8-78cebfb67d08
sh-5.1# 

wrote a small client to get the details, this is how it looks like

nixpanic commented 3 weeks ago

cidr:"10.244.0.1" is not in correct CIDR notation, it is a plain IP-address. Maybe call it differently, or make it cidr:"10.244.0.1/32".

Madhu-1 commented 3 weeks ago

cidr:"10.244.0.1" is not in correct CIDR notation, it is a plain IP-address. Maybe call it differently, or make it cidr:"10.244.0.1/32".

added /32 to make it CIDR.

Madhu-1 commented 3 weeks ago

hfs nodeplugin needs this NF server too

This is only for rbd now, we will extend it to cephfs later on if required.

nixpanic commented 3 weeks ago

@mergifyio rebase

mergify[bot] commented 3 weeks ago

rebase

✅ Branch has been successfully rebased

nixpanic commented 3 weeks ago

@mergifyio queue

mergify[bot] commented 3 weeks ago

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at *b4592a55ebe60dc81ba0a5d9a47fa83862a189a0*
ceph-csi-bot commented 3 weeks ago

/test ci/centos/k8s-e2e-external-storage/1.31

ceph-csi-bot commented 3 weeks ago

/test ci/centos/mini-e2e-helm/k8s-1.31

ceph-csi-bot commented 3 weeks ago

/test ci/centos/k8s-e2e-external-storage/1.29

ceph-csi-bot commented 3 weeks ago

/test ci/centos/mini-e2e/k8s-1.31

ceph-csi-bot commented 3 weeks ago

/test ci/centos/mini-e2e-helm/k8s-1.29

ceph-csi-bot commented 3 weeks ago

/test ci/centos/upgrade-tests-cephfs

ceph-csi-bot commented 3 weeks ago

/test ci/centos/k8s-e2e-external-storage/1.30

ceph-csi-bot commented 3 weeks ago

/test ci/centos/mini-e2e/k8s-1.29

ceph-csi-bot commented 3 weeks ago

/test ci/centos/upgrade-tests-rbd

ceph-csi-bot commented 3 weeks ago

/test ci/centos/mini-e2e-helm/k8s-1.30

ceph-csi-bot commented 3 weeks ago

/test ci/centos/mini-e2e/k8s-1.30

acolombier commented 2 weeks ago

I'm getting this error when trying to run the E2E. I assume librados-dev needs to be at a certain version. Could we make the required version explicit in the doc?

# github.com/ceph/ceph-csi/internal/util
internal/util/connection.go:162:17: cc.conn.GetAddrs undefined (type *rados.Conn has no field or method GetAddrs)
FAIL    github.com/ceph/ceph-csi/e2e [build failed]
FAIL
Madhu-1 commented 2 weeks ago

I'm getting this error when trying to run the E2E. I assume librados-dev needs to be at a certain version. Could we make the required version explicit in the doc?

# github.com/ceph/ceph-csi/internal/util
internal/util/connection.go:162:17: cc.conn.GetAddrs undefined (type *rados.Conn has no field or method GetAddrs)
FAIL    github.com/ceph/ceph-csi/e2e [build failed]
FAIL

The error seems to be coming from go-ceph. can you please specify which exact command is failing. https://docs.ceph.com/en/pacific/rados/api/librados/#c.rados_getaddrs is supported in older version as well. its not something new