microsoft / omi

Open Management Infrastructure
Other
367 stars 116 forks source link

selinux problem in omi on RHEL8 #680

Closed Klaas- closed 3 years ago

Klaas- commented 3 years ago

Hi, it seems the omi-logrotate selinux module is uninstalled after upgrading. I think this is a general build problem, scx has the same issue.

  Running scriptlet: omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/2
Removing selinux policy module for omi-logrotate ...
libsemanage.semanage_direct_remove_key: Removing last omi-logrotate module (no other omi-logrotate module exists at another priority).

How to reproduce: have a system without omi/scx

$ dnf install omi scx
Last metadata expiration check: 1:22:15 ago on Thu 11 Feb 2021 02:11:10 PM UTC.
Dependencies resolved.
=====================================================================================================================================================================================================================================================================================
 Package                                                    Architecture                                                  Version                                                           Repository                                                                          Size
=====================================================================================================================================================================================================================================================================================
Installing:
 omi                                                        x86_64                                                        1.6.6-0                                                           packages-microsoft-com-prod                                                        1.8 M
 scx                                                        x86_64                                                        1.6.6-0                                                           packages-microsoft-com-prod                                                        2.0 M

Transaction Summary
=====================================================================================================================================================================================================================================================================================
Install  2 Packages

Total download size: 3.8 M
Installed size: 11 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): scx-1.6.6-0.universal.x64.rpm                                                                                                                                                                                                                 9.6 MB/s | 2.0 MB     00:00
(2/2): omi-1.6.6-0.ssl_110.ulinux.x64.rpm                                                                                                                                                                                                            8.5 MB/s | 1.8 MB     00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                                 17 MB/s | 3.8 MB     00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                                                             1/1
  Running scriptlet: omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          1/2
Creating omi group ...
Creating omi service account ...

  Installing       : omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          1/2
  Running scriptlet: omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          1/2

************************************************************
* Warning: The certificate and keyfile were not generated  *
* since they already exist.                                *
************************************************************
2021-02-11 15:33:30 : Crontab not configured to update omi keytab automatically. Skip unconfigure
ktutil not found
Checking if cron is installed...
Checking if cron/crond service is started...
Set up a cron job to OMI logrotate every 15 minutes
System appears to have SELinux installed, attempting to install selinux policy module for logrotate
  Trying /usr/share/selinux/packages/omi-selinux/omi-logrotate.pp ...
  Trying /usr/share/selinux/packages/omi-selinux/omi-selinux.pp ...
  Labeling omi log files ...
Configuring OMI service ...
Created symlink /etc/systemd/system/multi-user.target.wants/omid.service → /usr/lib/systemd/system/omid.service.
Trying to start omi with systemctl
omi is started.

  Running scriptlet: scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/2
  Installing       : scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/2
  Running scriptlet: scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/2
Trying to stop omi with systemctl
omi is stopped.
Trying to start omi with systemctl
omi is started.
System appears to have SELinux installed, attempting to install selinux policy module for logrotate
  Trying /usr/share/selinux/packages/scxagent-logrotate/scxagent-logrotate.pp ...

  Verifying        : omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          1/2
  Verifying        : scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/2
Installed products updated.

Installed:
  omi-1.6.6-0.x86_64                                                                                                                        scx-1.6.6-0.x86_64

Complete!
$ semodule -l|grep -E 'scx|omi'
omi-logrotate
omi-selinux
scxagent-logrotate
$ dnf reinstall scx omi
Last metadata expiration check: 1:24:46 ago on Thu 11 Feb 2021 02:11:10 PM UTC.
Dependencies resolved.
=====================================================================================================================================================================================================================================================================================
 Package                                                    Architecture                                                  Version                                                           Repository                                                                          Size
=====================================================================================================================================================================================================================================================================================
Reinstalling:
 omi                                                        x86_64                                                        1.6.6-0                                                           packages-microsoft-com-prod                                                        1.8 M
 scx                                                        x86_64                                                        1.6.6-0                                                           packages-microsoft-com-prod                                                        2.0 M

Transaction Summary
=====================================================================================================================================================================================================================================================================================

Total download size: 3.8 M
Installed size: 11 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): scx-1.6.6-0.universal.x64.rpm                                                                                                                                                                                                                  12 MB/s | 2.0 MB     00:00
(2/2): omi-1.6.6-0.ssl_110.ulinux.x64.rpm                                                                                                                                                                                                             10 MB/s | 1.8 MB     00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                                 21 MB/s | 3.8 MB     00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                                                             1/1
  Running scriptlet: omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          1/4
Unconfiguring omid (systemd) service ...
Removed /etc/systemd/system/multi-user.target.wants/omid.service.

  Reinstalling     : omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          1/4
  Running scriptlet: omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          1/4

************************************************************
* Warning: The certificate and keyfile were not generated  *
* since they already exist.                                *
************************************************************
omi already configured
2021-02-11 15:36:02 : Crontab not configured to update omi keytab automatically. Skip unconfigure
ktutil not found
Checking if cron is installed...
Checking if cron/crond service is started...
Set up a cron job to OMI logrotate every 15 minutes
System appears to have SELinux installed, attempting to install selinux policy module for logrotate
  Trying /usr/share/selinux/packages/omi-selinux/omi-logrotate.pp ...
  Trying /usr/share/selinux/packages/omi-selinux/omi-selinux.pp ...
  Labeling omi log files ...
Configuring OMI service ...
Created symlink /etc/systemd/system/multi-user.target.wants/omid.service → /usr/lib/systemd/system/omid.service.
Trying to start omi with systemctl
omi is started.

  Running scriptlet: scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/4
  Reinstalling     : scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/4
  Running scriptlet: scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/4
Trying to stop omi with systemctl
omi is stopped.
Trying to start omi with systemctl
omi is started.
System appears to have SELinux installed, attempting to install selinux policy module for logrotate
  Trying /usr/share/selinux/packages/scxagent-logrotate/scxagent-logrotate.pp ...

  Running scriptlet: scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          3/4
  Cleanup          : scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          3/4
  Running scriptlet: scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          3/4
Trying to stop omi with systemctl
omi is stopped.
Trying to start omi with systemctl
omi is started.
Removing selinux policy module for scxagent-logrotate ...
libsemanage.semanage_direct_remove_key: Removing last scxagent-logrotate module (no other scxagent-logrotate module exists at another priority).

  Running scriptlet: omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          4/4
  Cleanup          : omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          4/4
  Running scriptlet: omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          4/4
Removing selinux policy module for omi-logrotate ...
libsemanage.semanage_direct_remove_key: Removing last omi-logrotate module (no other omi-logrotate module exists at another priority).

  Verifying        : omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          1/4
  Verifying        : omi-1.6.6-0.x86_64                                                                                                                                                                                                                                          2/4
  Verifying        : scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          3/4
  Verifying        : scx-1.6.6-0.x86_64                                                                                                                                                                                                                                          4/4
Installed products updated.

Reinstalled:
  omi-1.6.6-0.x86_64                                                                                                                        scx-1.6.6-0.x86_64

Complete!
$ semodule -l|grep -E 'scx|omi'
omi-selinux
Klaas- commented 3 years ago

I think this needs a change in https://github.com/microsoft/omi/blame/2cd827ba933a74374ca177007d4954aa8df493f3/Unix/installbuilder/datafiles/Linux.data#L366-L373 it needs to recognize if its being upgraded or uninstalled. I think this should also apply to rhel: https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/

Highlighting the author: @JumpingYang001

Klaas- commented 3 years ago

maybe you also want to change this in the script that builds the spec file from the data file to generalize the solution or move it into https://github.com/microsoft/omi/blame/2cd827ba933a74374ca177007d4954aa8df493f3/Unix/installbuilder/datafiles/Linux.data#L289

JumpingYang001 commented 3 years ago

@Klaas- thanks for reporting it! we will check the issue.

Klaas- commented 3 years ago

@JumpingYang001

The change you made somehow hasn't made it into the official rpm


$  cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.9 (Maipo)

$ rpm -qa|grep omi
omi-1.6.8-0.x86_64

$ rpm -qi omi
Name        : omi
Version     : 1.6.8
Release     : 0
Architecture: x86_64
Install Date: Fri 02 Apr 2021 03:45:05 AM CEST
Group       : System Environment/Daemons
Size        : 4608406
License     : MIT
Signature   : RSA/SHA256, Wed 31 Mar 2021 05:32:41 PM CEST, Key ID eb3e94adbe1229cf
Source RPM  : omi-1.6.8-0.src.rpm
Build Date  : Thu 14 Jan 2021 03:36:46 AM CET
Build Host  : osbld64-rhel5-01.scx.com
Relocations : (not relocatable)
Vendor      : Microsoft Corporation
Summary     : Open Management Infrastructure
Description :
omi server

$ rpm -qi --scripts omi
[...]
postuninstall scriptlet (using /bin/sh):
#!/bin/sh

if [ "$1" -ne 1 ]; then
    rm -f /opt/omi/lib/libcrypto* /opt/omi/lib/libssl* /opt/omi/lib/.libcrypto* /opt/omi/lib/.libssl*
    rmdir /opt/omi/lib > /dev/null 2>&1
    rmdir /opt/omi > /dev/null 2>&1

    # Clean up cron and logrotate
    rm -f /etc/cron.d/omilogrotate > /dev/null 2>&1
    rm -f /etc/logrotate.d/omi > /dev/null 2>&1

    egrep -q "^omiusers:" /etc/group
    if [ $? -eq 0 ]; then
        echo "Deleting omiusers group ..."
        groupdel omiusers
    fi
    egrep -q "^omi:" /etc/passwd
    if [ $? -eq 0 ]; then
       echo "Deleting omi service account ..."
           userdel omi
    fi
    egrep -q "^omi:" /etc/group
    if [ $? -eq 0 ]; then
        echo "Deleting omi group ..."
        groupdel omi
    fi
fi

if [ -e /usr/sbin/semodule ]; then
    if [ ! -z "$(/usr/sbin/semodule -l | grep omi-logrotate)" ]; then
        echo "Removing selinux policy module for omi-logrotate ..."
        /usr/sbin/semodule -r omi-logrotate
    fi
fi
exit 0

Greetings Klaas

Klaas- commented 3 years ago

side question: will this need more than one version upgrade to actually happen? on the next upgrade the current versions postuninstall script will be called right?

JumpingYang001 commented 3 years ago

@Klaas- yes, the upgrade uninstall script will be executed in next version.

Klaas- commented 3 years ago

@JumpingYang001 1) any idea why it's not in the official packages yet? 2) Do you plan to fix this by for example changing the upgrade procedure that happens on minor upgrades via waagent?

JumpingYang001 commented 3 years ago

@Klaas- it is already in official packages, and it is by design, since upgrade a package will use exist package's uninstall script and for current package's uninstall script will be executed in next upgrade.

Klaas- commented 3 years ago

@JumpingYang001 but it's not in the current packages scripts. 1.6.8-0 was tagged on github on Apr 9 and includes the fix. 1.6.8-0 from packages.microsoft.com was built on Thu 14 Jan 2021; so I am guessing you do not build from github sources and have some kind of own code staging for those packages, this seems to invite errors like this.

$ curl -O https://packages.microsoft.com/rhel/7/prod/omi-1.6.8-0.ssl_100.ulinux.x64.rpm
[...]
$ rpm -qp --scripts ./omi-1.6.8-0.ssl_100.ulinux.x64.rpm
[...]
postuninstall scriptlet (using /bin/sh):
#!/bin/sh

if [ "$1" -ne 1 ]; then
    rm -f /opt/omi/lib/libcrypto* /opt/omi/lib/libssl* /opt/omi/lib/.libcrypto* /opt/omi/lib/.libssl*
    rmdir /opt/omi/lib > /dev/null 2>&1
    rmdir /opt/omi > /dev/null 2>&1

    # Clean up cron and logrotate
    rm -f /etc/cron.d/omilogrotate > /dev/null 2>&1
    rm -f /etc/logrotate.d/omi > /dev/null 2>&1

    egrep -q "^omiusers:" /etc/group
    if [ $? -eq 0 ]; then
        echo "Deleting omiusers group ..."
        groupdel omiusers
    fi
    egrep -q "^omi:" /etc/passwd
    if [ $? -eq 0 ]; then
       echo "Deleting omi service account ..."
           userdel omi
    fi
    egrep -q "^omi:" /etc/group
    if [ $? -eq 0 ]; then
        echo "Deleting omi group ..."
        groupdel omi
    fi
fi

if [ -e /usr/sbin/semodule ]; then
    if [ ! -z "$(/usr/sbin/semodule -l | grep omi-logrotate)" ]; then
        echo "Removing selinux policy module for omi-logrotate ..."
        /usr/sbin/semodule -r omi-logrotate
    fi
fi
exit 0
JumpingYang001 commented 3 years ago

@Klaas- understand your question now, in fact, the 1.6.8-0 release tag wasn't updated by me... it was updated by other team member, yes, the tag is wrong... , and I check the real code for 1.6.8-0 should only includes these commits: https://github.com/microsoft/omi/commits/e6851ec20b00615d5fda8d3858cd5f142ed04528 .

Klaas- commented 3 years ago

So it will take another two releases of omi until this problem is addressed on it's own :) I am guessing it was a bad idea to wait for the fix, I am seeing a couple of multi-gb log files already, so I will clean this up in configuration management ... :)

JumpingYang001 commented 3 years ago

@Klaas- yeah, if you have urgent to fix it, you can manually do it at first, thanks for pointing out the issue.

Klaas- commented 3 years ago

@JumpingYang001 even loading the module is not enough :)

/etc/cron.daily/logrotate:

error: error accessing /var/opt/microsoft/omsconfig: Permission denied error: failed to rename /var/opt/microsoft/omsconfig/omsconfig.log to /var/opt/microsoft/omsconfig/omsconfig.log-20210812: Permission denied error: error accessing /var/opt/microsoft/omsconfig: Permission denied error: failed to rename /var/opt/microsoft/omsconfig/omsconfigdetailed.log to /var/opt/microsoft/omsconfig/omsconfigdetailed.log-20210812: Permission denied

Logrotate now has enough rights to access the file, but not enough to write the rotated log because it does not have rights on the directory :)

JumpingYang001 commented 3 years ago

@Klaas- /var/opt/microsoft/omsconfig path is another team's product directory, maybe you can contact them: https://github.com/Microsoft/PowerShell-DSC-for-Linux

Klaas- commented 3 years ago

Ah yes, sorry I see the policy is from https://github.com/microsoft/OMS-Agent-for-Linux/blob/master/installer/selinux/omsagent-logrotate.fc , I'll raise this issue there

Klaas- commented 3 years ago

It seems like there is an issue about this in that repo already, https://github.com/microsoft/OMS-Agent-for-Linux/issues/781#issuecomment-639801741