search

ceph / ceph-helm

Curated applications for Kubernetes
Apache License 2.0
108 stars 36 forks source link

pod "ceph-osd-dev-" blocked after the k8s recovered from a crash #69

Open wadefelix opened 6 years ago

wadefelix commented 6 years ago

https://github.com/ceph/ceph-helm/blob/743a7441ba4361866a6e017b4f8fa5c15e34e640/ceph/ceph/templates/bin/_osd_disk_prepare.sh.tpl#L30

I installed a ceph cluster using the ceph-helm, but my kubernetes cluster crashed for some reason. After I restored the k8s cluster, all the pods recovered excluding the ceph-osd-dev- pods.

The initContainer osd-prepare-pod logs:

2018-08-25 04:37:41  /start_osd.sh: Checking if it belongs to this cluster
++ echo 2712
+ tmp_osd_mount=/var/lib/ceph/tmp/2712/
+ mkdir -p /var/lib/ceph/tmp/2712/
+ mount /dev/loop01 /var/lib/ceph/tmp/2712/
mount: special device /dev/loop01 does not exist

I checked the devices in /dev/ , 

$ ls /dev/loop*
/dev/loop0  /dev/loop0p1  /dev/loop0p2  /dev/loop-control

I suspect the /dev/loop0p1 is the device wanted, so I make a symbol link , and IT WORKS!

I think there is something wrong with it. The ceph deamon image is

registry.docker-cn.com/ceph/daemon                                         tag-build-master-luminous-ubuntu-16.04   c48fa6936ae5        6 months ago        445 MB