avocado-framework / avocado-vt

Avocado VT Plugin
https://avocado-vt.readthedocs.org/
Other
83 stars 241 forks source link

utils-test:Add running htx-cmdline stress_type and fix invalid sequence #3976

Open misanjumn opened 3 weeks ago

misanjumn commented 3 weeks ago

utils-test:Add running htx-cmdline stress_type and fix invalid sequence

This patch can be used to run htx mdts stress_type using htxcmdline tool Additionally, this patch fixes DeprecationWarning of invalid escape sequence 'x' W605

This patch introduces the following changes:

  1. Allow running of different stress types in different vms based on stresstype%s parameter if provided in the cfg file, where %s is the name of the vm. This will allow us to introduce additional types of stress in the future. If the stresstype%s paramter is not provided, then the default stress_type will be used.

  2. Define a new function load_htxstress_tool() to run htxcmdline tool in the guest. This function will be called whenever stress_type_vmname = htxcmdline_in_vms. The user can provide the link to the .rpm/.deb package and the function will handle the installation, followed by the running of the htxcmdline tool by using the htxcmdline_args parameter, all provided by user in the cfg file.

Signed-off-by: Misbah Anjum N misanjum@linux.vnet.ibm.com

misanjumn commented 3 weeks ago

Example config parameters

Running stress on 3 vms: vm1: stress vm2: htxcmdline vm3: stress

stress_args_vm1 = --cpu 4 --io 4 --vm 2 --vm-bytes 256M

stress_type_vm2 = htxcmdline_in_vms
stress_cmds_htxcmdline_vm2 = "htxcmdline"
htxcmdline_args_vm2 = "-run -mdt mdt.mem"
download_url_htxcmdline_vm2 = "example/url/htx.rpm"
make_cmds_htxcmdline_vm2 = "rpm -i htx.rpm"

stress_args_vm3 = --cpu 2 --io 4

Resultant test execution with the above config parameters

call function for vm1

def load_stress_tool(self):
.....

call funcation for vm2

    def load_htxstress_tool(self):
        """
        load stress tool in guest
        """
        # install htx strss tool
        remove_old_package = "cd " + self.dst_path + " && rm -rf " + self.base_name
        get_stress = "cd " + self.dst_path + " && wget " + self.download_url
        if "rpm" in self.base_name:
            install_stress = "cd " + self.dst_path + " && rpm -i " + self.base_name
        elif "deb" in self.base_name:
            install_stress = "cd " + self.dst_path + " && dpkg -i " + self.base_name
        else:
            raise exceptions.TestError("Could not install htx stress tool")
        LOG.info("Installing htx rpm/deb in the guest")
        self.cmd(remove_old_package)
        self.cmd(get_stress)
        self.cmd(install_stress)

        # start htx stress
        launch_cmds = "nohup %s %s > /dev/null &" % (self.stress_cmds, self.stress_args)
        LOG.info("Launch stress with command: %s", launch_cmds)
        try:
            self.cmd_launch(launch_cmds)
            # The background process sometimes does not return to
            # terminate, if timeout, send a blank line afterward
        except aexpect.ShellTimeoutError:
            self.cmd_launch("")

        # wait for stress to start and then check, if not raise TestError
        if not utils_misc.wait_for(
            self.app_running,
            self.stress_wait_for_timeout,
            first=2.0,
            text="wait for stress app to start",
            step=1.0,
        ):
            raise exceptions.TestError("Stress app does not " "running as expected")

call function for vm3


def load_stress_tool(self):
.....
misanjumn commented 3 weeks ago

Running multivm_stress.py test

 (1/1) type_specific.io-github-autotest-libvirt.multivm_cpustress.custom_host_events.custom_vm_events: STARTED
 (1/1) type_specific.io-github-autotest-libvirt.multivm_cpustress.custom_host_events.custom_vm_events:  PASS (189.14 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

Debug log of vm1 stress

[stdlog] 2024-08-20 14:04:33,260 aexpect.remote remote           L1027 INFO | Copy file from /var/tmp/avocado_mwx3p5_e/stress-1.0.4 to ***:/home, elapsed time: 0.9505650997161865
[stdlog] 2024-08-20 14:04:33,260 avocado.virttest.utils_test __init__         L2471 INFO | installing the stress with cd /home/stress-1.0.4/;./configure && make install
[stdlog] 2024-08-20 14:04:33,261 aexpect.client client           L1199 DEBUG| Sending command: cd /home/stress-1.0.4/;./configure && make install
[stdlog] 2024-08-20 14:04:38,088 aexpect.client client           L1199 DEBUG| Sending command: echo $?
[stdlog] 2024-08-20 14:04:38,189 aexpect.client client           L1238 DEBUG| Sending command (safe): cd /home/stress-1.0.4/
[stdlog] 2024-08-20 14:04:38,291 avocado.virttest.utils_test __init__         L2278 INFO | Launch stress with command: nohup stress --cpu 4 --io 4 --vm 2 --vm-bytes 256M > /dev/null &
[stdlog] 2024-08-20 14:04:38,291 aexpect.client client           L1238 DEBUG| Sending command (safe): nohup stress --cpu 4 --io 4 --vm 2 --vm-bytes 256M > /dev/null &
[stdlog] 2024-08-20 14:04:40,907 avocado.virttest.utils_misc utils_misc       L0578 DEBUG| wait for stress app to start (2.000092 secs)
[stdlog] 2024-08-20 14:04:40,907 aexpect.client client           L1199 DEBUG| Sending command: pidof -s stress
[stdlog] 2024-08-20 14:04:41,040 aexpect.client client           L1199 DEBUG| Sending command: echo $?

Debug log of vm2 htxcmdline

[stdlog] 2024-08-20 14:04:44,354 avocado.virttest.utils_test __init__         L2309 INFO | Installing htx rpm/deb in the guest
[stdlog] 2024-08-20 14:04:44,355 aexpect.client client           L1199 DEBUG| Sending command: cd /home && rm -rf htx.rpm
[stdlog] 2024-08-20 14:04:44,459 aexpect.client client           L1199 DEBUG| Sending command: echo $?
[stdlog] 2024-08-20 14:04:44,575 aexpect.client client           L1199 DEBUG| Sending command: cd /home && wget example/url/htx.rpm
[stdlog] 2024-08-20 14:05:44,585 aexpect.client client           L1199 DEBUG| Sending command: echo $?
[stdlog] 2024-08-20 14:05:44,635 aexpect.client client           L1199 DEBUG| Sending command: cd /home && rpm -i htx.rpm
[stdlog] 2024-08-20 14:05:51,435 aexpect.client client           L1199 DEBUG| Sending command: echo $?
[stdlog] 2024-08-20 14:05:51,538 avocado.virttest.utils_test __init__         L2316 INFO | Launch stress with command: nohup htxcmdline -run -mdt mdt.mem > /dev/null &
[stdlog] 2024-08-20 14:05:51,539 aexpect.client client           L1238 DEBUG| Sending command (safe): nohup htxcmdline -run -mdt mdt.mem > /dev/null &
[stdlog] 2024-08-20 14:05:54,150 avocado.virttest.utils_misc utils_misc       L0578 DEBUG| wait for stress app to start (2.000075 secs)
[stdlog] 2024-08-20 14:05:54,150 aexpect.client client           L1199 DEBUG| Sending command: pidof -s htxcmdline
[stdlog] 2024-08-20 14:05:54,264 aexpect.client client           L1199 DEBUG| Sending command: echo $?

Debug log of vm3 stress

[stdlog] 2024-08-20 14:05:59,294 aexpect.remote remote           L1027 INFO | Copy file from /var/tmp/avocado_mwx3p5_e/stress-1.0.4 to ***:/home, elapsed time: 1.695960283279419
[stdlog] 2024-08-20 14:05:59,294 avocado.virttest.utils_test __init__         L2471 INFO | installing the stress with cd /home/stress-1.0.4/;./configure && make install
[stdlog] 2024-08-20 14:05:59,294 aexpect.client client           L1199 DEBUG| Sending command: cd /home/stress-1.0.4/;./configure && make install
[stdlog] 2024-08-20 14:06:05,273 aexpect.client client           L1199 DEBUG| Sending command: echo $?
[stdlog] 2024-08-20 14:06:05,375 aexpect.client client           L1238 DEBUG| Sending command (safe): cd /home/stress-1.0.4/
[stdlog] 2024-08-20 14:06:05,493 avocado.virttest.utils_test __init__         L2278 INFO | Launch stress with command: nohup stress --cpu 2 --io 4 > /dev/null &
[stdlog] 2024-08-20 14:06:05,493 aexpect.client client           L1238 DEBUG| Sending command (safe): nohup stress --cpu 2 --io 4 > /dev/null &
[stdlog] 2024-08-20 14:06:08,111 avocado.virttest.utils_misc utils_misc       L0578 DEBUG| wait for stress app to start (2.000103 secs)
[stdlog] 2024-08-20 14:06:08,111 aexpect.client client           L1199 DEBUG| Sending command: pidof -s stress
misanjumn commented 3 weeks ago

Fixing invalid sequence

Before patch

% inspekt style utils_test.py
PEP8 disabled: E501,E265,W601,E402
utils_test.py:374:36: W605 invalid escape sequence '\d'
utils_test.py:374:41: W605 invalid escape sequence '\d'
utils_test.py:377:38: W605 invalid escape sequence '\s'
utils_test.py:377:40: W605 invalid escape sequence '\d'
utils_test.py:767:42: W605 invalid escape sequence '\w'
utils_test.py:787:46: W605 invalid escape sequence '\w'
utils_test.py:1398:49: W605 invalid escape sequence '\d'
utils_test.py:1401:59: W605 invalid escape sequence '\d'
utils_test.py:1957:36: W605 invalid escape sequence '\s'
utils_test.py:1974:36: W605 invalid escape sequence '\s'
Style: utils_test.py FAIL 
PEP8 compliance FAIL

After patch

% inspekt style utils_test.py
PEP8 disabled: E501,E265,W601,E402
PEP8 compliance check PASS
PraveenPenguin commented 2 weeks ago

@clebergnu can you please help us to trigger CI for this PR

misanjumn commented 2 weeks ago

Updated the code based on the CI checks. Thanks.