[6.16.z] disable pytest plugin autoloading when running FAM tests

Satellite-QE commented 2 days ago

Cherrypick of PR: https://github.com/SatelliteQE/robottelo/pull/16975

Problem Statement

pytest by default tries to load all plugins it finds on a system. pulpcore ships with own pytest plugins, but we do not install their dependencies (as we don't want to run pulpcore tests), which leads to pytest failing to load those plugins and breaking overall execution:

Traceback (most recent call last):
  File "/usr/bin/pytest-3.11", line 33, in <module>
    sys.exit(load_entry_point('pytest==7.2.0', 'console_scripts', 'pytest')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 190, in console_main
    code = main()
           ^^^^^^
  File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 148, in main
    config = _prepareconfig(args, plugins)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.11/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
    config: Config = outcome.get_result()
                     ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1058, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1346, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1229, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
    plugin = ep.load()
             ^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/usr/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
    exec(co, module.__dict__)
  File "/usr/lib/python3.11/site-packages/pulp_ansible/pytest_plugin.py", line 3, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

Solution

Disable the autoloading by setting the PYTEST_DISABLE_PLUGIN_AUTOLOAD environment variable.

Related Issues

PRT test Cases example

trigger: test-robottelo pytest: tests/foreman/sys/test_fam.py

Satellite-QE commented 2 days ago

trigger: test-robottelo
pytest: tests/foreman/sys/test_fam.py

Satellite-QE commented 2 days ago

PRT Result

Build Number: 9430
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/sys/test_fam.py --external-logging
Test Result : =========== 14 failed, 96 passed, 340 warnings in 5095.68s (1:24:55) ===========

SatelliteQE / robottelo