search

ceph / ceph-iscsi-cli

NOTICE: moved to https://github.com/ceph/ceph-iscsi
GNU General Public License v3.0
25 stars 25 forks source link

create snapshot of rbd image failed with errno 30 #71

Open zhuozh opened 6 years ago

zhuozh commented 6 years ago

When a rbd image is mapped as a LUN, the tcmu-runner will hold the image's exclusive-lock. If we create a snapshot of this image, it will fail with errno 30 (EROFS: Read-only file system) Is there any plan or suggestion on how to support rbd snapshot without unbind the LUN, for a reseaon that we may running a VM on this LUN. VM be stopped or crashed is unacceptable when creating snapshot.

70 is releated to this issue.

mikechristie commented 6 years ago

Hey,

We have sort of discussed in this internally, but any low level details.

I think we would like a gwcli command that calls into ceph-iscsi-config to coordinate the tcmu-runner access and do the ceph/rbd lib calls to do the snapshot.

If you have any ideas, post them!

dillaman commented 6 years ago

@zhuozh There shouldn't be any restriction for creating a snapshot when the image's exclusive lock is owned by another client. The client requesting the snapshot create will send an RPC message to the lock owner to create the snapshot on its behalf. Do you have a reproducer for this behavior? I just tried it in my environment and it worked.

zhuozh commented 6 years ago

@dillaman My expression is a little misleading. Actually the read only error happened when rolling back an image to a snapshot.

  1. Create a LUN, mkfs and mount it on initiator node.
  2. Write some data into mounted device.
  3. Create a snapshot using rbd command
  4. iscsiadm logout, unbind clientlun through gwcli.
  5. Roll back to a snapshot, it fails with read-only error.
dillaman commented 6 years ago

@zhuozh That was originally a deliberate decision on the part of librbd since we really didn't expect a valid use-case where you should rollback with an alive client. If tcmu-runner knew when the last client disconnected, it could release the lock (but I have a feeling it doesn't know). Instead, as an alternative, we could add a --force optional to the rbd CLI to break the lock before a rollback.

mikechristie commented 6 years ago

Adding @pcuzner

I think Paul wanted to add gwcli commands for snapshots that coordinated with the target/runner to handle this case for the user.

dillaman commented 6 years ago

See new PR https://github.com/ceph/ceph-iscsi-cli/pull/93