Closed yaelmi3 closed 1 month ago
I still don't understand this issue, but I run into it from time to time in different environments. I'm using this workaround to manually force the verbose
option.
from pathlib import Path
from _pytest.config import PytestPluginManager, Config, default_plugins
def run_pytest_cli(args, plugins):
# The majority of this function is copied from Pytest internals
pluginmanager = PytestPluginManager()
config = Config(
pluginmanager,
invocation_params=Config.InvocationParams(
args=args,
plugins=plugins,
dir=Path.cwd(),
),
)
# Here is the patch I've added to Pytest's source code to get around
# https://github.com/pytest-dev/pytest/issues/9422
config.option.verbose = True
pluginmanager.consider_preparse(args, exclude_only=True)
for default_plugin in default_plugins:
pluginmanager.import_plugin(default_plugin)
pluginmanager = config.pluginmanager
for plugin in plugins:
pluginmanager.register(plugin)
config = pluginmanager.hook.pytest_cmdline_parse(
pluginmanager=pluginmanager, args=args
)
exit_code = config.hook.pytest_cmdline_main(config=config)
return exit_code
# e.g.
if __name__ == "__main__":
failure_collector_plugin = FailureCollectorPlugin()
exit_code = run_pytest_cli(
args=["-p", "no:terminal"],
plugins=[failure_collector_plugin],
)
Replication of the internals is a last resort i strongly recommend against using
That being said, the use of the option should be changed
However it should also be noted that disabling built in plugins is potentially dangerous
I recommend taking a look at how xdist and sugar manage things
I'm also running into this and have used a hack to get around the verbose problems, but in recent versions the assertion rewriter plugin has been failing if the terminal plugin is disabled.
I wonder if the thing to do is to add a way to disable all output from the terminal reporter. The use case for this is when tests are run in a subprocess launched by an IDE or other tool and test success / failure is not reported through terminal output.
The simple fix would be to add a default to the getoption
call, so the error will not happen anymore in case the option is not defined, for example config.getoption("verbose")
to config.getoption("verbose", 0)
. While might seem like a workaround, I think it is reasonable and simple enough to get this in.
Would somebody like to give this a try?
Hello @nicoddemus , I'm new to OSS and would love to take this up. Thanks!
Hi @ka28kumar, that's great to hear. No need for permission to get started, feel free to fork the project, and submit a PR (even if incomplete).
Hi @nicoddemus ,
Rather than replacing every call of config.getoption("verbose")
to config.getoption("verbose", 0)
, which also includes some examples in the docs. I'd prefer if we can add a default value of 0 to the verbose
option. Before that, I'm not able to reproduce the issue in question. I took the example above, and executed it with both -p no:terminal
and without it, and it executes the same.
I believe its a good idea to unify all options of the built-in plugins so disabling plugins won't break CLI args and option expectations for the built-in plugins
I'd prefer if we can add a default value of 0 to the verbose option.
What do you mean, add the default to the verbose definition? Note that's already there:
The problem is actually that with -p no:terminal
the option is not even defined, because the terminal plugin does not get loaded.
I believe its a good idea to unify all options of the built-in plugins so disabling plugins won't break CLI args and option expectations for the built-in plugins
Interesting idea, for some builtin plugins this seems like a good option, not sure the same is true for every builtin plugin (for example stepwise
, pastebin
, etc).
Execute
pytest
without the terminal plugin, i.e.pytest -p no:terminal -s
, when the test contains an assertion failure, for example:An exception is issued: (https://github.com/pytest-dev/pytest/blob/main/src/_pytest/assertion/util.py#L161)
Additional reference to
config.option.verbose
: https://github.com/pytest-dev/pytest/blob/main/src/_pytest/assertion/truncate.py#L29The
verbose
parameter is set alongside with the Terminal plugin (https://github.com/pytest-dev/pytest/blob/main/src/_pytest/terminal.py#L144), but apparently is used by the assertion utils as well. Possible solution, movingverbosity
setting from terminal pytest adoption to runner pytest_adoption , since the verbosity setting affects modules outside the terminal outputOur specific use case: Disable Terminal Plugin and forward results using dedicated logger. We achieve it by implementing the following:
Then we execute pytest with terminal plugin disabled. The problem occurs upon assertion failures and currently our workaround is setting
config.option.verbose
during the thepytest_configure
on our end.