Add support for cephadm instead of ceph-ansible

[EmilienM]

Ceph has given up ceph-ansible and therefore TripleO had to remove its support going forward.

This patch will detect if ceph-ansible can still be used and if not it'll configure ceph using cephadm.

From a user standpoint, dev-install still offers 3 options:

• Disable the management of Ceph from dev-install; if users don't want Ceph or if they want to use extra_heat_params.
• Let cephadm to install Ceph OSDs on available disks that aren't rotational (for performance purposes). This is different from ceph-ansible where we could choose the disks, but one can achieve it via option 1 if they don't have SSD/NVME disks.
• Let dev-install creating LVM resources on a loop device and let cephadm to use these volumes to configure Ceph. Note that cephadm doesn't need the DB volume but this shouldn't matter anyway, as the data volume should be big enough to run the workloads (volumes, vms, etc), since it's 96% of the total size of the loop device.

In other words, this change is backward compatible except if you deployed Ceph with ceph_devices that are rotational; then no OSD will be deployed and cephadm will crash. In that case it's suggested to go with option 1 previously described, but dev-install maintainers don't suggest to install Ceph on these disks because of performance impact.

[EmilienM]

I tested it against real disks and also with the default (loop device) and it worked fine. I think this is good to go unless there are concerns about the gotchas in the commit message.

[mdbooth]

/lgtm

[mandre]

Looks good. Perhaps we should print a warning when ceph_devices is set and we detect that we'll use cephadm to deploy ceph, to let the user know this variable WILL NOT be honored.

[EmilienM]

Looks good. Perhaps we should print a warning when ceph_devices is set and we detect that we'll use cephadm to deploy ceph, to let the user know this variable WILL NOT be honored.

done

[mandre]

Looks good. Perhaps we should print a warning when ceph_devices is set and we detect that we'll use cephadm to deploy ceph, to let the user know this variable WILL NOT be honored.

done

Perhaps I've completely misunderstood the gotcha with ceph_devices, but it seems to me that it's not used unless the environment uses ceph-ansible. In this case the warning you've added doesn't really address my concern. I want to avoid:

• users setting ceph_devices like before and expecting it to work with cephadm
• the cephadm crash reported in the PR description (how is it possible if ceph_devices is ignored with cephadm?)