odf-cli app is not found test_debug_verbosity_of_ceph_component - Githubissues

red-hat-storage / ocs-ci

https://ocs-ci.readthedocs.io/en/latest/

MIT License

108 stars 166 forks source link

odf-cli app is not found test_debug_verbosity_of_ceph_component #10445

Open DanielOsypenko opened 2 months ago

DanielOsypenko commented 2 months ago

Tests test_debug_verbosity_of_ceph_component are failing to run odf-cli commands

[2024-09-06T11:17:53.489Z] 07:17:52 - MainThread - ocs_ci.utility.utils - [32mINFO[0m  - Executing command: /home/jenkins/workspace/qe-deploy-ocs-cluster/ocs-ci/data/odf-cli --kubeconfig /home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig  set ceph log-level osd crush 50
[2024-09-06T11:17:53.489Z] 07:17:52 - MainThread - ocs_ci.framework.pytest_customization.reports - [32mINFO[0m  - duration reported by tests/functional/odf-cli/test_debug_verbocity_of_ceph_component.py::TestDebugVerbosityOfCephComponents::test_debug_verbosity_of_ceph_components[osd-crush] immediately after test execution: 9.93
[2024-09-06T11:17:53.489Z] [31mFAILED[0m
[2024-09-06T11:17:53.489Z] _ TestDebugVerbosityOfCephComponents.test_debug_verbosity_of_ceph_components[osd-crush] _
[2024-09-06T11:17:53.489Z] 
[2024-09-06T11:17:53.489Z] self = <test_debug_verbocity_of_ceph_component.TestDebugVerbosityOfCephComponents object at 0x7f60f8955be0>
[2024-09-06T11:17:53.489Z] service = 'osd', subsystem = 'crush'
[2024-09-06T11:17:53.489Z] 
[2024-09-06T11:17:53.489Z]     @pytest.mark.polarion_id("OCS-5417")
[2024-09-06T11:17:53.489Z]     @pytest.mark.parametrize(
[2024-09-06T11:17:53.489Z]         argnames=["service", "subsystem"],
[2024-09-06T11:17:53.489Z]         argvalues=[("osd", "crush"), ("mds", "crush"), ("mon", "crush")],
[2024-09-06T11:17:53.489Z]     )
[2024-09-06T11:17:53.489Z]     def test_debug_verbosity_of_ceph_components(self, service, subsystem):
[2024-09-06T11:17:53.489Z]         """
[2024-09-06T11:17:53.489Z]         Test setting the debug verbosity of Ceph components using ODF CLI.
[2024-09-06T11:17:53.489Z]         Steps:
[2024-09-06T11:17:53.489Z]             1. Set log-level using ODF cli tool for services {mon, mds, osd }
[2024-09-06T11:17:53.489Z]             2. Verify log-level from the ceph toolbox pod
[2024-09-06T11:17:53.489Z]             3. Test Overriding log level with different value.
[2024-09-06T11:17:53.489Z]             4. Test Setting up log level to upper limit 99.
[2024-09-06T11:17:53.489Z]             5. Test Setting up log level to lower limit 0.
[2024-09-06T11:17:53.489Z]             6. Test Setting up log level beyond limit.
[2024-09-06T11:17:53.489Z]     
[2024-09-06T11:17:53.489Z]         """
[2024-09-06T11:17:53.489Z]         log_levels = (50, 99, 0, 100)
[2024-09-06T11:17:53.489Z]     
[2024-09-06T11:17:53.489Z]         for log_level in log_levels:
[2024-09-06T11:17:53.489Z]             # Setting up and verifying the log level value with the odf CLI tool
[2024-09-06T11:17:53.489Z]             log.info(
[2024-09-06T11:17:53.489Z]                 f"Setting log level to {log_level} for service: {service}, subsystem: {subsystem}"
[2024-09-06T11:17:53.489Z]             )
[2024-09-06T11:17:53.489Z]     
[2024-09-06T11:17:53.489Z]             if log_level > 99:
[2024-09-06T11:17:53.489Z]                 with pytest.raises(CommandFailed):
[2024-09-06T11:17:53.489Z]                     odf_cli_set_log_level(service, log_level, subsystem)
[2024-09-06T11:17:53.489Z]                     log.info("Log level beyond the limit was not set as expected.")
[2024-09-06T11:17:53.489Z]             else:
[2024-09-06T11:17:53.489Z] >               assert odf_cli_set_log_level(service, log_level, subsystem)
[2024-09-06T11:17:53.489Z] 
[2024-09-06T11:17:53.489Z] [1m[31mtests/functional/odf-cli/test_debug_verbocity_of_ceph_component.py[0m:45: 
[2024-09-06T11:17:53.489Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2024-09-06T11:17:53.489Z] [1m[31mocs_ci/helpers/helpers.py[0m:4741: in odf_cli_set_log_level
[2024-09-06T11:17:53.489Z]     return exec_cmd(cmd, use_shell=True)
[2024-09-06T11:17:53.489Z] [1m[31mocs_ci/utility/utils.py[0m:673: in exec_cmd
[2024-09-06T11:17:53.489Z]     completed_process = subprocess.run(
[2024-09-06T11:17:53.489Z] [1m[31m/usr/lib64/python3.9/subprocess.py[0m:505: in run
[2024-09-06T11:17:53.489Z]     with Popen(*popenargs, **kwargs) as process:
[2024-09-06T11:17:53.489Z] [1m[31m/usr/lib64/python3.9/subprocess.py[0m:951: in __init__
[2024-09-06T11:17:53.489Z]     self._execute_child(args, executable, preexec_fn, close_fds,
[2024-09-06T11:17:53.489Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[2024-09-06T11:17:53.489Z] 
[2024-09-06T11:17:53.489Z] self = <Popen: returncode: 255 args: ['/home/jenkins/workspace/qe-deploy-ocs-cluste...>
[2024-09-06T11:17:53.489Z] args = ['/home/jenkins/workspace/qe-deploy-ocs-cluster/ocs-ci/data/odf-cli', '--kubeconfig', '/home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig', 'set', 'ceph', 'log-level', ...]
[2024-09-06T11:17:53.489Z] executable = b'/home/jenkins/workspace/qe-deploy-ocs-cluster/ocs-ci/data/odf-cli'
[2024-09-06T11:17:53.489Z] preexec_fn = None, close_fds = True, pass_fds = (), cwd = None, env = None
[2024-09-06T11:17:53.489Z] startupinfo = None, creationflags = 0, shell = False, p2cread = 26
[2024-09-06T11:17:53.489Z] p2cwrite = 27, c2pread = 28, c2pwrite = 29, errread = 30, errwrite = 31
[2024-09-06T11:17:53.489Z] restore_signals = True, gid = None, gids = None, uid = None, umask = -1
[2024-09-06T11:17:53.489Z] start_new_session = False
[2024-09-06T11:17:53.489Z] 
[2024-09-06T11:17:53.489Z]     def _execute_child(self, args, executable, preexec_fn, close_fds,
[2024-09-06T11:17:53.489Z]                        pass_fds, cwd, env,
[2024-09-06T11:17:53.489Z]                        startupinfo, creationflags, shell,
[2024-09-06T11:17:53.489Z]                        p2cread, p2cwrite,
[2024-09-06T11:17:53.489Z]                        c2pread, c2pwrite,
[2024-09-06T11:17:53.489Z]                        errread, errwrite,
[2024-09-06T11:17:53.489Z]                        restore_signals,
[2024-09-06T11:17:53.489Z]                        gid, gids, uid, umask,
[2024-09-06T11:17:53.489Z]                        start_new_session):
[2024-09-06T11:17:53.489Z]         """Execute program (POSIX version)"""
[2024-09-06T11:17:53.489Z]     
[2024-09-06T11:17:53.489Z]         if isinstance(args, (str, bytes)):
[2024-09-06T11:17:53.489Z]             args = [args]
[2024-09-06T11:17:53.489Z]         elif isinstance(args, os.PathLike):
[2024-09-06T11:17:53.489Z]             if shell:
[2024-09-06T11:17:53.489Z]                 raise TypeError('path-like args is not allowed when '
[2024-09-06T11:17:53.489Z]                                 'shell is true')
[2024-09-06T11:17:53.489Z]             args = [args]
[2024-09-06T11:17:53.489Z]         else:
[2024-09-06T11:17:53.489Z]             args = list(args)
[2024-09-06T11:17:53.489Z]     
[2024-09-06T11:17:53.489Z]         if shell:
[2024-09-06T11:17:53.489Z]             # On Android the default shell is at '/system/bin/sh'.
[2024-09-06T11:17:53.489Z]             unix_shell = ('/system/bin/sh' if
[2024-09-06T11:17:53.489Z]                       hasattr(sys, 'getandroidapilevel') else '/bin/sh')
[2024-09-06T11:17:53.489Z]             args = [unix_shell, "-c"] + args
[2024-09-06T11:17:53.489Z]             if executable:
[2024-09-06T11:17:53.489Z]                 args[0] = executable
[2024-09-06T11:17:53.489Z]     
[2024-09-06T11:17:53.489Z]         if executable is None:
[2024-09-06T11:17:53.489Z]             executable = args[0]
[2024-09-06T11:17:53.489Z]     
[2024-09-06T11:17:53.490Z]         sys.audit("subprocess.Popen", executable, args, cwd, env)
[2024-09-06T11:17:53.490Z]     
[2024-09-06T11:17:53.490Z]         if (_USE_POSIX_SPAWN
[2024-09-06T11:17:53.490Z]                 and os.path.dirname(executable)
[2024-09-06T11:17:53.490Z]                 and preexec_fn is None
[2024-09-06T11:17:53.490Z]                 and not close_fds
[2024-09-06T11:17:53.490Z]                 and not pass_fds
[2024-09-06T11:17:53.490Z]                 and cwd is None
[2024-09-06T11:17:53.490Z]                 and (p2cread == -1 or p2cread > 2)
[2024-09-06T11:17:53.490Z]                 and (c2pwrite == -1 or c2pwrite > 2)
[2024-09-06T11:17:53.490Z]                 and (errwrite == -1 or errwrite > 2)
[2024-09-06T11:17:53.490Z]                 and not start_new_session
[2024-09-06T11:17:53.490Z]                 and gid is None
[2024-09-06T11:17:53.490Z]                 and gids is None
[2024-09-06T11:17:53.490Z]                 and uid is None
[2024-09-06T11:17:53.490Z]                 and umask < 0):
[2024-09-06T11:17:53.490Z]             self._posix_spawn(args, executable, env, restore_signals,
[2024-09-06T11:17:53.490Z]                               p2cread, p2cwrite,
[2024-09-06T11:17:53.490Z]                               c2pread, c2pwrite,
[2024-09-06T11:17:53.490Z]                               errread, errwrite)
[2024-09-06T11:17:53.490Z]             return
[2024-09-06T11:17:53.490Z]     
[2024-09-06T11:17:53.490Z]         orig_executable = executable
[2024-09-06T11:17:53.490Z]     
[2024-09-06T11:17:53.490Z]         # For transferring possible exec failure from child to parent.
[2024-09-06T11:17:53.490Z]         # Data format: "exception name:hex errno:description"
[2024-09-06T11:17:53.490Z]         # Pickle is not used; it is complex and involves memory allocation.
[2024-09-06T11:17:53.490Z]         errpipe_read, errpipe_write = os.pipe()
[2024-09-06T11:17:53.490Z]         # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
[2024-09-06T11:17:53.490Z]         low_fds_to_close = []
[2024-09-06T11:17:53.490Z]         while errpipe_write < 3:
[2024-09-06T11:17:53.490Z]             low_fds_to_close.append(errpipe_write)
[2024-09-06T11:17:53.490Z]             errpipe_write = os.dup(errpipe_write)
[2024-09-06T11:17:53.490Z]         for low_fd in low_fds_to_close:
[2024-09-06T11:17:53.490Z]             os.close(low_fd)
[2024-09-06T11:17:53.490Z]         try:
[2024-09-06T11:17:53.490Z]             try:
[2024-09-06T11:17:53.490Z]                 # We must avoid complex work that could involve
[2024-09-06T11:17:53.490Z]                 # malloc or free in the child process to avoid
[2024-09-06T11:17:53.490Z]                 # potential deadlocks, thus we do all this here.
[2024-09-06T11:17:53.490Z]                 # and pass it to fork_exec()
[2024-09-06T11:17:53.490Z]     
[2024-09-06T11:17:53.490Z]                 if env is not None:
[2024-09-06T11:17:53.490Z]                     env_list = []
[2024-09-06T11:17:53.490Z]                     for k, v in env.items():
[2024-09-06T11:17:53.490Z]                         k = os.fsencode(k)
[2024-09-06T11:17:53.490Z]                         if b'=' in k:
[2024-09-06T11:17:53.490Z]                             raise ValueError("illegal environment variable name")
[2024-09-06T11:17:53.490Z]                         env_list.append(k + b'=' + os.fsencode(v))
[2024-09-06T11:17:53.490Z]                 else:
[2024-09-06T11:17:53.490Z]                     env_list = None  # Use execv instead of execve.
[2024-09-06T11:17:53.490Z]                 executable = os.fsencode(executable)
[2024-09-06T11:17:53.490Z]                 if os.path.dirname(executable):
[2024-09-06T11:17:53.490Z]                     executable_list = (executable,)
[2024-09-06T11:17:53.490Z]                 else:
[2024-09-06T11:17:53.490Z]                     # This matches the behavior of os._execvpe().
[2024-09-06T11:17:53.490Z]                     executable_list = tuple(
[2024-09-06T11:17:53.490Z]                         os.path.join(os.fsencode(dir), executable)
[2024-09-06T11:17:53.490Z]                         for dir in os.get_exec_path(env))
[2024-09-06T11:17:53.490Z]                 fds_to_keep = set(pass_fds)
[2024-09-06T11:17:53.490Z]                 fds_to_keep.add(errpipe_write)
[2024-09-06T11:17:53.490Z]                 self.pid = _posixsubprocess.fork_exec(
[2024-09-06T11:17:53.490Z]                         args, executable_list,
[2024-09-06T11:17:53.490Z]                         close_fds, tuple(sorted(map(int, fds_to_keep))),
[2024-09-06T11:17:53.490Z]                         cwd, env_list,
[2024-09-06T11:17:53.490Z]                         p2cread, p2cwrite, c2pread, c2pwrite,
[2024-09-06T11:17:53.490Z]                         errread, errwrite,
[2024-09-06T11:17:53.490Z]                         errpipe_read, errpipe_write,
[2024-09-06T11:17:53.490Z]                         restore_signals, start_new_session,
[2024-09-06T11:17:53.490Z]                         gid, gids, uid, umask,
[2024-09-06T11:17:53.490Z]                         preexec_fn)
[2024-09-06T11:17:53.490Z]                 self._child_created = True
[2024-09-06T11:17:53.490Z]             finally:
[2024-09-06T11:17:53.490Z]                 # be sure the FD is closed no matter what
[2024-09-06T11:17:53.490Z]                 os.close(errpipe_write)
[2024-09-06T11:17:53.490Z]     
[2024-09-06T11:17:53.490Z]             self._close_pipe_fds(p2cread, p2cwrite,
[2024-09-06T11:17:53.490Z]                                  c2pread, c2pwrite,
[2024-09-06T11:17:53.490Z]                                  errread, errwrite)
[2024-09-06T11:17:53.490Z]     
[2024-09-06T11:17:53.490Z]             # Wait for exec to fail or succeed; possibly raising an
[2024-09-06T11:17:53.490Z]             # exception (limited in size)
[2024-09-06T11:17:53.490Z]             errpipe_data = bytearray()
[2024-09-06T11:17:53.490Z]             while True:
[2024-09-06T11:17:53.490Z]                 part = os.read(errpipe_read, 50000)
[2024-09-06T11:17:53.490Z]                 errpipe_data += part
[2024-09-06T11:17:53.490Z]                 if not part or len(errpipe_data) > 50000:
[2024-09-06T11:17:53.490Z]                     break
[2024-09-06T11:17:53.490Z]         finally:
[2024-09-06T11:17:53.490Z]             # be sure the FD is closed no matter what
[2024-09-06T11:17:53.490Z]             os.close(errpipe_read)
[2024-09-06T11:17:53.490Z]     
[2024-09-06T11:17:53.490Z]         if errpipe_data:
[2024-09-06T11:17:53.490Z]             try:
[2024-09-06T11:17:53.490Z]                 pid, sts = os.waitpid(self.pid, 0)
[2024-09-06T11:17:53.490Z]                 if pid == self.pid:
[2024-09-06T11:17:53.490Z]                     self._handle_exitstatus(sts)
[2024-09-06T11:17:53.490Z]                 else:
[2024-09-06T11:17:53.490Z]                     self.returncode = sys.maxsize
[2024-09-06T11:17:53.490Z]             except ChildProcessError:
[2024-09-06T11:17:53.490Z]                 pass
[2024-09-06T11:17:53.490Z]     
[2024-09-06T11:17:53.490Z]             try:
[2024-09-06T11:17:53.490Z]                 exception_name, hex_errno, err_msg = (
[2024-09-06T11:17:53.490Z]                         errpipe_data.split(b':', 2))
[2024-09-06T11:17:53.490Z]                 # The encoding here should match the encoding
[2024-09-06T11:17:53.490Z]                 # written in by the subprocess implementations
[2024-09-06T11:17:53.490Z]                 # like _posixsubprocess
[2024-09-06T11:17:53.490Z]                 err_msg = err_msg.decode()
[2024-09-06T11:17:53.490Z]             except ValueError:
[2024-09-06T11:17:53.490Z]                 exception_name = b'SubprocessError'
[2024-09-06T11:17:53.490Z]                 hex_errno = b'0'
[2024-09-06T11:17:53.490Z]                 err_msg = 'Bad exception data from child: {!r}'.format(
[2024-09-06T11:17:53.490Z]                               bytes(errpipe_data))
[2024-09-06T11:17:53.490Z]             child_exception_type = getattr(
[2024-09-06T11:17:53.490Z]                     builtins, exception_name.decode('ascii'),
[2024-09-06T11:17:53.490Z]                     SubprocessError)
[2024-09-06T11:17:53.490Z]             if issubclass(child_exception_type, OSError) and hex_errno:
[2024-09-06T11:17:53.490Z]                 errno_num = int(hex_errno, 16)
[2024-09-06T11:17:53.490Z]                 child_exec_never_called = (err_msg == "noexec")
[2024-09-06T11:17:53.490Z]                 if child_exec_never_called:
[2024-09-06T11:17:53.490Z]                     err_msg = ""
[2024-09-06T11:17:53.490Z]                     # The error must be from chdir(cwd).
[2024-09-06T11:17:53.490Z]                     err_filename = cwd
[2024-09-06T11:17:53.490Z]                 else:
[2024-09-06T11:17:53.490Z]                     err_filename = orig_executable
[2024-09-06T11:17:53.490Z]                 if errno_num != 0:
[2024-09-06T11:17:53.490Z]                     err_msg = os.strerror(errno_num)
[2024-09-06T11:17:53.490Z] >               raise child_exception_type(errno_num, err_msg, err_filename)
[2024-09-06T11:17:53.490Z] [1m[31mE               FileNotFoundError: [Errno 2] No such file or directory: '/home/jenkins/workspace/qe-deploy-ocs-cluster/ocs-ci/data/odf-cli'[0m
[2024-09-06T11:17:53.490Z] 
[2024-09-06T11:17:53.490Z] [1m[31m/usr/lib64/python3.9/subprocess.py[0m:1837: FileNotFoundError
[2024-09-06T11:17:53.490Z]

DanielOsypenko commented 1 month ago

new failure - https://ocs4-jenkins-csb-odf-qe.apps.ocp-c1.prod.psi.redhat.com/job/qe-deploy-ocs-cluster/42316/ @paraggit

paraggit commented 1 month ago

Fixed via PR https://github.com/red-hat-storage/ocs-ci/pull/10627