When calling host.package().is_installed in a system using RPMs (in our case, RHEL), and the actual RPM database is corrupted, the command returns True, but the package could not even be queried to know if it's installed or not.
We got a false positive in this test case:
[gw3] linux -- Python 3.9.16 /opt/app-root/bin/python3.9
[gw3] linux -- Python 3.9.16 /opt/app-root/bin/python3.9[gw3] linux -- Python 3.9.16 /opt/app-root/bin/python3.9
self = <test_aws.TestsAWS object at 0x7fba46f33c70>
host = <testinfra.host.Host paramiko://<masked>.us-gov-east-1.compute.amazonaws.com>
@pytest.mark.run_on(['rhel'])
def test_unwanted_packages_are_not_present(self, host):
"""
Some pkgs are not required in EC2.
BugZilla 1888695, 2075815
"""
unwanted_pkgs = [
'aic94xx-firmware', 'alsa-firmware', 'alsa-lib', 'alsa-tools-firmware',
'ivtv-firmware', 'iwl1000-firmware', 'iwl100-firmware', 'iwl105-firmware',
'iwl135-firmware', 'iwl2000-firmware', 'iwl2030-firmware', 'iwl3160-firmware',
'iwl3945-firmware', 'iwl4965-firmware', 'iwl5000-firmware', 'iwl5150-firmware',
'iwl6000-firmware', 'iwl6000g2a-firmware', 'iwl6000g2b-firmware', 'iwl6050-firmware',
'iwl7260-firmware', 'libertas-sd8686-firmware', 'libertas-sd8787-firmware', 'libertas-usb8388-firmware',
'firewalld', 'biosdevname', 'plymouth', 'iprutils', 'rng-tools', 'qemu-guest-agent'
]
system_release = version.parse(host.system_info.release)
# BugZilla 1888695
if version.parse('8.3') > system_release >= version.parse('8.0'):
unwanted_pkgs.remove('rng-tools')
if system_release < version.parse('8.5'):
unwanted_pkgs.remove('qemu-guest-agent')
if test_lib.is_rhel_sap(host):
# In RHEL SAP images, alsa-lib is allowed
unwanted_pkgs.remove('alsa-lib')
if test_lib.is_rhel_high_availability(host):
unwanted_pkgs.append('rh-amazon-rhui-client')
found_pkgs = [
pkg for pkg in unwanted_pkgs if host.package(pkg).is_installed]
> assert len(
found_pkgs) == 0, f'Found unexpected packages installed: {", ".join(found_pkgs)}'
E AssertionError: Found unexpected packages installed: alsa-lib
E assert 1 == 0
E + where 1 = len(['alsa-lib'])
test_suite/cloud/test_aws.py:233: AssertionError[gw3] linux -- Python 3.9.16 /opt/app-root/bin/python3.9
is_installed fails since it cannot check for the package existence as rpm command crashed unexpectedly due to a DB error, or any other reason. And this reason is thrown as an a exception so the user know what happened.
Actual behavior
is_installed returns True when there is a rpm command failure and it is returning 1.
Suggestion / Proposal
Make RPM database integrity check, or at least improve the is_installed function to throw an exception when the rpm query could not be done (considering that other errors can actually also return 1 when running rpm command).
We can check the stdout to really know that the package is not installed:
package <pkg> is not installed (return code 1)
If not matching, we can print the actual stdout + stderr, or the CommandOutput object itself, such as it is being done in the rest of the project.
Summary
When calling
host.package().is_installed
in a system using RPMs (in our case, RHEL), and the actual RPM database is corrupted, the command returns True, but the package could not even be queried to know if it's installed or not.We got a false positive in this test case:
Source: https://github.com/osbuild/cloud-image-val/blob/5e2d95961646dd66e5962c9f8c65133713b1ee40/test_suite/cloud/test_aws.py#L199
I found this issue debugging the tool and adding more information in some of our test results, as you can see below:
Source: https://github.com/osbuild/cloud-image-val/blob/5e2d95961646dd66e5962c9f8c65133713b1ee40/test_suite/cloud/test_aws.py#L320
Expected behavior
is_installed fails since it cannot check for the package existence as rpm command crashed unexpectedly due to a DB error, or any other reason. And this reason is thrown as an a exception so the user know what happened.
Actual behavior
is_installed returns True when there is a rpm command failure and it is returning 1.
Suggestion / Proposal
Make RPM database integrity check, or at least improve the is_installed function to throw an exception when the rpm query could not be done (considering that other errors can actually also return 1 when running rpm command).
We can check the stdout to really know that the package is not installed:
package <pkg> is not installed
(return code 1)If not matching, we can print the actual stdout + stderr, or the CommandOutput object itself, such as it is being done in the rest of the project.