ceph-ansible issue that impacts linode

[bengland2]

This ceph-ansible issue means that you can't use monitor_address_block ansible var to set monitor addresses. Has anyone else encountered this?This seems like an absolute requirement for linode because of the use of multiple ethernet addresses associated with eth0, only 1 of which is on the same subnet 192.168.128.0/17 for all linodes in my cluster.

[root@li289-169 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 66.228.39.169  netmask 255.255.255.0  broadcast 66.228.39.255
        inet6 2600:3c03::f03c:91ff:febd:18b7  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::f03c:91ff:febd:18b7  prefixlen 64  scopeid 0x20<link>
        ether f2:3c:91:bd:18:b7  txqueuelen 1000  (Ethernet)
...
[root@li289-169 ~]# ip a 
...
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether f2:3c:91:bd:18:b7 brd ff:ff:ff:ff:ff:ff
    inet 66.228.39.169/24 brd 66.228.39.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.152.194/17 brd 192.168.255.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2600:3c03::f03c:91ff:febd:18b7/64 scope global noprefixroute dynamic 
       valid_lft 2592000sec preferred_lft 604800sec
    inet6 fe80::f03c:91ff:febd:18b7/64 scope link 
       valid_lft forever preferred_lft forever
...

I worked around it by disabling the check in ceph-ansible that prevents monitor_address_block from being used.

diff --git a/roles/ceph-common/tasks/checks/check_mandatory_vars.yml b/roles/ceph-common/tasks/checks/check_mandatory_vars.yml
index 3525e0f3..5f203d1e 100644
--- a/roles/ceph-common/tasks/checks/check_mandatory_vars.yml
+++ b/roles/ceph-common/tasks/checks/check_mandatory_vars.yml
@@ -75,14 +75,6 @@
   tags:
     - package-install

-- name: make sure monitor_interface or monitor_address is defined
-  fail:
-    msg: "you must set monitor_interface or monitor_address"
-  when:
-    - monitor_interface == 'interface'
-    - monitor_address == "0.0.0.0"
-    - mon_group_name in group_names
-
 - name: make sure radosgw_interface or radosgw_address is defined
   fail:
     msg: "you must set radosgw_interface or radosgw_address"

[batrick]

Has anyone else encountered this?

Only other person I'm aware testing this is @fullerdj.

I worked around it by disabling the check in ceph-ansible that prevents monitor_address_block from being used.

Unfortunately the ceph-ansible folks aren't actively using monitor_address_block in their testing so every now and then stuff like this happens. (They almost removed it a few months ago! We must remain vigilent ;)

[bengland2]

I think monitor_interface and monitor_address should be REMOVED from ceph-ansible and only monitor_address_block should be allowed. And the default for monitor_address_block should be whatever the public_network subnet is. That would be simple, wouldn't it?

[batrick]

It would make sense to me, yes.

[batrick]

ceph-ansible is dead, long live cephadm!