Use LVM storage operator for dynamic PV provisioning

[anjannath]

Currently we are using the hostpath-provisioner for dynamic pv creation, however since it has limited functionality work is being done in https://github.com/crc-org/snc/pull/877, https://github.com/crc-org/snc/pull/869 and #4114 to install and setup lvm-operator

We need to make a decision whether to make the lvm-operator the default one and remove hostpath-provisioner, the behavior of expanding disk space using the config settingdisk-size will change if we move to lvm-operator as in that case disk-size will refer to the root partition size

to increase the space for PVs user has to use the config option persistent-volume-size

while talking with @praveenkumar he suggested the following to avoid any behavior change for users who don't want to use lvm-operator and are happy with hostpath-provisioner:

First method

1. Introduce a new config ~enable-lvm-operator~ use-lvm-operator with default value false to allow users to make a choice of which operator they want to use
2. When this config is enabled/set to true the LVMCluster CR will be created and the extra partition will be used for PV
3. When this config is disabled/set to false the root partition is extended to take up the space from the extra partition (and optionally also delete the operator installation or scale down the deployments as the operatorgroup and subscription objects are created during snc)

Second method

1. Introduce a new config enable-lvm-operator with default value false to allow users to make a choice of which operator they want to use
2. When this config is enabled/set to true the extra partition will not be merged with the root partition and user has to follow the guide on openshift docs to setup/install the LVM operator on their own
3. When this config is disabled/set to false the extra partition is merged into the root partition

[praveenkumar]

done in # and # to install and setup

Looks like you forget to attach the issue number.

I personally think second method is much better because it will educate user about how it is enabled in a OpenShift cluster and also we don't have to worry about the changes which might happen during major release.

[cfergeau]

I don't think we should keep both lvm-operator and hostpath-provisioner. We pick one, and we maintain that. Easier for testing, easier for users who don't care about storage operators, ...

[anjannath]

supporting both makes it complicated for the user, and as using topolvm needs to alter the disk layout for creating a separate partition to be used, so it also changes the existing behavior of disk-resizing.

currently crc config disk-size 35 would increase the root partition size by 4gb, but when we use topolvm since the root partition is before the topolvm partition we cannot easily resize the root partition, (in case of microshift the root partition is also a lvm logical volume)

we could look at moving the topolvm partition to the end of the disk after a disk-size increase and then increase the root parition (seems using sfdisk this is possible, but requires a restart of the crc vm) but is there any use case where users would need to increase the root partition? (note: the image-registry is also using a pvc for storage so images pulled will be stored in the topolvm partition)

[anjannath]

is there any use case where users would need to increase the root partition? (note: the image-registry is also using a pvc for storage so images pulled will be stored in the topolvm partition)

pulled images are stored in the dir /var/lib/containers/storage and when running containers it uses the location /var/run/containers/storage which are both in the root partition, so there might be need to increase the root partition, furthermore and there's also emptyDir type volumes for ephemeral storage which uses /tmp location