This feature implements PR https://github.com/codedellemc/scaleio-framework/issues/82 Imperative approach to deploying ScaleIO. This is a significant feature and therefore required a lot of restructuring to the code base. If the declarative approach is not used, the prior method for adding ScaleIO nodes to the cluster is still supported (but has been enhanced - explained further down in this PR).
Imperative Deployment:
To declare what Block/Storage Devices (if any) contributes to what StoragePools and what StoragePools contribute to which ProtectionDomains, the following Mesos Agent nodes attributes (read: http://mesos.apache.org/documentation/latest/attributes-resources/) must be defined: scaleio-sds-domains, scaleio-sds-[domain], and scaleio-sds-[pool]. An example of this is given below:
To declare what Mesos Agent nodes consume storage out of which ProtectionDomain/StoragePool combination, the following Mesos Agent nodes attributes must be defined: scaleio-sdc-domains and scaleio-sdc-[domain]. (Note the sds vs sdc difference). IMPORTANT: There is a limitation currently in rexray such that you can only consume a single ProtectionDomain/StoragePool combination. What ever the first attribute pair discover by the Mesos Agent will be the pair selected. If no sdc is defined, the first attribute pair from the sds attributes will be used. Else the default in the Configuration flag will be used (default values are both default for ProtectionDomain and StoragePool). Example below:
In order to implement this feature, state of the ScaleIO cluster and the nodes had to be introduced and therefore libkv is being used to store ScaleIO metadata. The configuration of this metadata store by default uses the zookeeper configuration which happens to be picked up automatically. Otherwise an external KeyValue store can be used (consul, zookeeper, etcd).
Another feature that has been enhanced is that when new nodes are brought online, storage devices will be add to the ScaleIO cluster based on the method being used.
If the Imperative approach is used, only devices defined by the user will be added.
The old automatic method is used, any available block devices that are currently not being used (ie has a filesystem on it) will be added to the default domain/pool.
This feature implements PR https://github.com/codedellemc/scaleio-framework/issues/82 Imperative approach to deploying ScaleIO. This is a significant feature and therefore required a lot of restructuring to the code base. If the declarative approach is not used, the prior method for adding ScaleIO nodes to the cluster is still supported (but has been enhanced - explained further down in this PR).
Imperative Deployment:
To declare what Block/Storage Devices (if any) contributes to what StoragePools and what StoragePools contribute to which ProtectionDomains, the following Mesos Agent nodes attributes (read: http://mesos.apache.org/documentation/latest/attributes-resources/) must be defined: scaleio-sds-domains, scaleio-sds-[domain], and scaleio-sds-[pool]. An example of this is given below:
/etc/mesos-slave/attributes/scaleio-sds-domains = mydomain,myotherdomain /etc/mesos-slave/attributes/scaleio-sds-mydomain1 = mypool /etc/mesos-slave/attributes/scaleio-sds-myotherdomain = saltwaterpool /etc/mesos-slave/attributes/scaleio-sds-mypool = /dev/xvdf /etc/mesos-slave/attributes/scaleio-sds-saltwaterpool = /dev/xvdg
To declare what Mesos Agent nodes consume storage out of which ProtectionDomain/StoragePool combination, the following Mesos Agent nodes attributes must be defined: scaleio-sdc-domains and scaleio-sdc-[domain]. (Note the sds vs sdc difference). IMPORTANT: There is a limitation currently in rexray such that you can only consume a single ProtectionDomain/StoragePool combination. What ever the first attribute pair discover by the Mesos Agent will be the pair selected. If no sdc is defined, the first attribute pair from the sds attributes will be used. Else the default in the Configuration flag will be used (default values are both default for ProtectionDomain and StoragePool). Example below:
/etc/mesos-slave/attributes/scaleio-sdc-domains = mydomain /etc/mesos-slave/attributes/scaleio-sdc-mydomain1 = mypool
Other Enhancements
In order to implement this feature, state of the ScaleIO cluster and the nodes had to be introduced and therefore libkv is being used to store ScaleIO metadata. The configuration of this metadata store by default uses the zookeeper configuration which happens to be picked up automatically. Otherwise an external KeyValue store can be used (consul, zookeeper, etcd).
Another feature that has been enhanced is that when new nodes are brought online, storage devices will be add to the ScaleIO cluster based on the method being used.