ceph-iscsi: add erasure pool support

[lxbsz]

Erasure coded pools do not support omap, will just store the data in ec pool, and their metadata will be in 'rbd' replicated pool.

Signed-off-by: Xiubo Li xiubli@redhat.com

[lxbsz]

@idryomov

All my tests worked well till now. Please review, thanks.

[lxbsz]

Add one ecpool= option when creating the disk:

/> disks/ create image=block14 pool=datapool ecpool=ecpool size=1M

Without the ecpool= specified, it will use the pool= to save both metadata and data. Or it will use the pool= to save the metadata and use the ecpool= to save the data.

For the image with the ecpool= specified it will display the disks like: <replicated pool name>/<erasure pool name>/<image name>

[root@node01 ceph-iscsi]# gwcli ls
o- / ......................................................................................................................... [...]
  o- cluster ......................................................................................................... [Clusters: 1]
  | o- ceph .......................................................................................................... [HEALTH_WARN]
  |   o- pools .......................................................................................................... [Pools: 6]
  |   | o- cephfs.a.data ......................................................... [(x3), Commit: 0.00Y/29819642K (0%), Used: 0.00Y]
  |   | o- cephfs.a.meta ....................................................... [(x3), Commit: 0.00Y/29819642K (0%), Used: 134000b]
  |   | o- datapool ............................................................... [(x3), Commit: 3M/29819642K (0%), Used: 418943b]
  |   | o- device_health_metrics ................................................. [(x3), Commit: 0.00Y/29819642K (0%), Used: 0.00Y]
  |   | o- ecpool ................................................................. [(3+1), Commit: 0.00Y/59639284K (0%), Used: 24K]
  |   | o- rbd ................................................................. [(x3), Commit: 0.00Y/29819642K (0%), Used: 103304b]
  |   o- topology ................................................................................................ [OSDs: 3,MONs: 3]
  o- disks .......................................................................................................... [3M, Disks: 3]
  | o- datapool .................................................................................................... [datapool (3M)]
  |   o- block14 ............................................................................ [datapool/ecpool/block14 (Online, 1M)]
  |   o- block15 ................................................................................... [datapool/block15 (Online, 1M)]
  |   o- block16 ............................................................................ [datapool/ecpool/block16 (Online, 1M)]
  o- iscsi-targets ............................................................................... [DiscoveryAuth: None, Targets: 1]
    o- iqn.2003-01.com.redhat.iscsi-gw:ceph-gw ........................................................... [Auth: None, Gateways: 2]
      o- disks .......................................................................................................... [Disks: 3]
      | o- datapool/block15 ................................................................................ [Owner: node01, Lun: 0]
      | o- datapool/ecpool/block14 ......................................................................... [Owner: node02, Lun: 1]
      | o- datapool/ecpool/block16 ......................................................................... [Owner: node01, Lun: 2]
      o- gateways ............................................................................................ [Up: 2/2, Portals: 2]
      | o- node01 ............................................................................................ [172.16.219.128 (UP)]
      | o- node02 ............................................................................................ [172.16.219.138 (UP)]
      o- host-groups .................................................................................................. [Groups : 0]
      o- hosts ....................................................................................... [Auth: ACL_ENABLED, Hosts: 0]

[lxbsz]

@idryomov

Have fixed them all, the gwcli ls will be like:

[root@node02 ceph-iscsi]# gwcli ls
o- / ......................................................................................................................... [...]
  o- cluster ......................................................................................................... [Clusters: 1]
  | o- ceph .......................................................................................................... [HEALTH_WARN]
  |   o- pools .......................................................................................................... [Pools: 6]
  |   | o- cephfs.a.data ......................................................... [(x3), Commit: 0.00Y/28722104K (0%), Used: 0.00Y]
  |   | o- cephfs.a.meta ....................................................... [(x3), Commit: 0.00Y/28722104K (0%), Used: 134000b]
  |   | o- datapool ........................................................ [(x3), Commit: 0.00Y/28722104K (0%), Used: 3228247703b]
  |   | o- device_health_metrics ................................................. [(x3), Commit: 0.00Y/28722104K (0%), Used: 0.00Y]
  |   | o- ecpool .............................................................. [(2+2), Commit: 0.00Y/57444208K (0%), Used: 30788K]
  |   | o- rbd ................................................................. [(x3), Commit: 36M/28722104K (0%), Used: 12500238b]
  |   o- topology ................................................................................................ [OSDs: 3,MONs: 3]
  o- disks ......................................................................................................... [36M, Disks: 3]
  | o- rbd ............................................................................................................. [rbd (36M)]
  |   o- blockX0 ...................................................................................... [rbd/blockX0 (Unknown, 12M)]
  |   o- blockX1 ...................................................................................... [rbd/blockX1 (Unknown, 12M)]
  |   o- blockX2 ............................................................................... [rbd+ecpool/blockX2 (Unknown, 12M)]
  o- iscsi-targets ............................................................................... [DiscoveryAuth: None, Targets: 1]
    o- iqn.2003-01.com.redhat.iscsi-gw:ceph-igw0 ......................................................... [Auth: None, Gateways: 2]
      o- disks .......................................................................................................... [Disks: 2]
      | o- rbd/blockX0 ..................................................................................... [Owner: node01, Lun: 0]
      | o- rbd/blockX2 ..................................................................................... [Owner: node02, Lun: 1]
      o- gateways ............................................................................................ [Up: 2/2, Portals: 2]
      | o- node01 ............................................................................................ [172.16.219.128 (UP)]
      | o- node02 ............................................................................................ [172.16.219.138 (UP)]
      o- host-groups .................................................................................................. [Groups : 0]
      o- hosts ....................................................................................... [Auth: ACL_ENABLED, Hosts: 1]
        o- iqn.1994-05.com.redhat:client .................................................... [LOGGED-IN, Auth: None, Disks: 2(24M)]
          o- lun 0 ............................................................................... [rbd/blockX0(12M), Owner: node01]
          o- lun 1 ............................................................................... [rbd/blockX2(12M), Owner: node02]

Please take a look of this, all the Rest APIs still the same as before except add a datapool=<name> parameter for some of them.

I am still testing this version, not finished yet.

BTW, do you think we need to display the <rep>+<ec>/image under the iscsi-targets ?

[lxbsz]

Till now all my tests work well.

[idryomov]

BTW, do you think we need to display the <rep>+<ec>/image under the iscsi-targets ?

Probably not.

[lxbsz]

I don't think you meant to include gwcli/storage.py.orig file in the PR. I can't see gwcli/storage.py diff because of that.

Removed. Thanks.

[lxbsz]

@idryomov Please take a look when you are free, thanks.