New subvolume code fails to generate fstab entry on SES6 GMC2 (and probably GM too!).

[tuxwielder]

Description of Issue/Question

Since GMC2 the btrfs volume step as documented in the official deployment and admin guides fails (i.e. the instruction to run "root@master # salt-run state.orch ceph.migrate.subvolume"). I'm too frustrated to create a separate bug-report on the production! documentation being wrong :).

It fails because since commit c0eef0534c81e441578e5c7cfb682c3a395124d0 this logic has been moved to 'srv/salt/ceph/subvolume'

The README for this, is not packaged into the GMC2 RPM's, but reading the instructions from the Github repository, we are supposed to run:

  # salt '*' saltutil.sync_all
  # salt 'minion*' state.apply ceph.subvolume

As a preparatory step on the monitor minions.

The latter fails on our SLES15SP1/SES6 GMC2 (and probably SES6 GM!) deployments, because the code in '/srv/salt/ceph/subvolume/default.sls' assumes a default label in fstab ('LABEL=ROOT'), whereas our SLES15SP1/SES6 GMC2 installation uses UUID as labels (e.g. UUID=e0f0dd5d-4559-46fa-8894-c76da5fb359e /var btrfs subvol=/@/var 0 0 ).

Therefore the fstab is never adopted properly.

Setup

Just install SLES15SP1 with SES6 GMC2 and follow the new instructions.

Steps to Reproduce Issue

  sds-adm1:/# salt 'sds-mon*' state.apply ceph.subvolume
  sds-mon1.XXXXX:
    Name: btrfs subvolume create /var/lib/ceph - Function: cmd.run - Result: Changed Started: - 14:17:03.400141 Duration: 18.126 ms
    Name: sed -i 's!LABEL=ROOT /var btrfs defaults,subvol=@/var 0 0!&\
  LABEL=ROOT /var/lib/ceph btrfs defaults,subvol=@/var/lib/ceph 0 0!' /etc/fstab - Function: 
  cmd.run - Result: Changed Started: - 14:17:03.418499 Duration: 12.234 ms
  ----------
            ID: mount
      Function: cmd.run
          Name: mount /var/lib/ceph
        Result: False
       Comment: Command "mount /var/lib/ceph" run
       Started: 14:17:03.430969
      Duration: 14.736 ms
       Changes:
                ----------
                pid:
                    26163
                retcode:
                    1
                stderr:
                    mount: /var/lib/ceph: can't find in /etc/fstab.
                stdout:

  Summary for sds-mon1.XXXXX
  ------------
  Succeeded: 2 (changed=3)
  Failed:    1
 ------------
  Total states run:     3
  Total run time:  45.096 ms
  ....

Versions Report

(Provided by running: salt-run deepsea.version rpm -qi salt-minion rpm -qi salt-master )

  sds-adm1:~ # salt-run deepsea.version
  0.9.22+git.0.29e84a00e
  sds-adm1:~ # rpm -qi salt-minion
 Name        : salt-minion
  Version     : 2019.2.0
  Release     : 4.4
  Architecture: x86_64
  Install Date: Wed Jul  3 11:03:46 2019
  Group       : System/Management
  Size        : 41019
  License     : Apache-2.0
  Signature   : RSA/SHA256, Mon May 13 14:44:42 2019, Key ID 70af9e8139db7c82
  Source RPM  : salt-2019.2.0-4.4.src.rpm
 Build Date  : Mon May 13 14:40:28 2019
  Build Host  : sheep65
  Relocations : (not relocatable)
  Packager    : https://www.suse.com/
 Vendor      : SUSE LLC <https://www.suse.com/>
  URL         : http://saltstack.org/
 Summary     : The client component for Saltstack
  Description :
 Salt minion is queried and controlled from the master.
 Listens to the salt master and execute the commands.
  Distribution: SUSE Linux Enterprise 15
  sds-adm1:~ # rpm -qi salt-master
  Name        : salt-master
  Version     : 2019.2.0
  Release     : 4.4
  Architecture: x86_64
  Install Date: Wed Jul  3 09:58:24 2019
   Group       : System/Management
  Size        : 2936818
  License     : Apache-2.0
  Signature   : RSA/SHA256, Mon May 13 14:44:42 2019, Key ID 70af9e8139db7c82
  Source RPM  : salt-2019.2.0-4.4.src.rpm
  Build Date  : Mon May 13 14:40:28 2019
  Build Host  : sheep65
  Relocations : (not relocatable)
  Packager    : https://www.suse.com/
  Vendor      : SUSE LLC <https://www.suse.com/>
  URL         : http://saltstack.org/
  Summary     : The management component of Saltstack with zmq protocol supported
  Description :
  The Salt master is the central server to which all minions connect.
  Enabled commands to remote systems to be called in parallel rather
  than serially.
  Distribution: SUSE Linux Enterprise 15

[tuxwielder]

The following seems to fix our issue for us:

--- srv/salt/ceph/subvolume/default.sls.org 2019-07-03 16:24:24.393357287 +0200
+++ srv/salt/ceph/subvolume/default.sls 2019-07-03 16:42:44.248197060 +0200
@@ -8,7 +8,7 @@
 {# sed is easier to explain/debug than file.replace #}
 fstab:
   cmd.run:
-    - name: "sed -i 's!LABEL=ROOT /var btrfs defaults,subvol=@/var 0 0!&\\\nLABEL=ROOT /var/lib/ceph btrfs defaults,subvol=@/var/lib/ceph 0 0!' /etc/fstab"
+    - name: "sed -i 's!\\(^[^ ]*\\).*/var .*.*$!&\\n\\1  /var/lib/ceph           btrfs  subvol=@/var/lib/ceph         0  0!' /etc/fstab"
     - unless: "grep -q subvol=@/var/lib/ceph /etc/fstab"
     - failhard: True