2024.03.02: pytest fails in 4 units

kloczek commented 5 months ago

Current Behavior

Here is pytest summary output:

```console =========================== short test summary info ============================ SKIPPED [1] tests/test_command.py:62: Needs git SKIPPED [1] tests/test_command.py:362: Only run this test on Windows. SKIPPED [1] tests/test_command.py:384: Only run this test on Windows. SKIPPED [1] tests/test_tox_to_nox.py:284: Not supported in tox 4. SKIPPED [1] tests/test_virtualenv.py:154: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:183: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:195: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:209: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:239: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:245: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:262: Missing uv command. SKIPPED [1] tests/test_virtualenv.py:423: Missing uv command. SKIPPED [1] tests/test_virtualenv.py:455: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:468: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:479: Missing uv command. SKIPPED [1] tests/test_virtualenv.py:479: Missing conda command. SKIPPED [1] tests/test_virtualenv.py:500: Missing uv command. SKIPPED [1] tests/test_virtualenv.py:526: Missing uv command. SKIPPED [1] tests/test_virtualenv.py:599: Python 2.7 unsupported for virtualenv>=20.22.0 XPASS tests/test_logger.py::test_no_color_timestamp[color] FAILED tests/test_sessions.py::TestSessionRunner::test__create_venv_options[nox.virtualenv.CondaEnv.create-conda-CondaEnv] FAILED tests/test_virtualenv.py::test_stale_environment[venv-virtualenv-True] FAILED tests/test_virtualenv.py::test_create_reuse_venv_environment - nox.com... FAILED tests/test_virtualenv.py::test_create_venv_backend - nox.command.Comma... ============ 4 failed, 520 passed, 19 skipped, 1 xpassed in 16.41s ============= ```

In attachment is full pytest output python-nox.FAIL.txt in which is possible to find few call traces

Expected Behavior

pytest should not fail.

Steps To Reproduce

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

python3 -sBm build -w --no-isolation
because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
install .whl file in </install/prefix> using installer module
run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Environment

- OS: Linux x86/64
- Python: 3.9.18
- Nox: 2024.03.02

Anything else?

List of installed modules in build env:

```console Package Version ----------------------------- ----------- alabaster 0.7.16 argcomplete 3.2.3 Babel 2.14.0 build 1.1.1 cachetools 5.3.3 chardet 5.2.0 charset-normalizer 3.3.2 colorama 0.4.6 colorlog 6.8.2 distro 1.9.0 docutils 0.20.1 editables 0.5 exceptiongroup 1.1.3 filelock 3.13.1 hatchling 1.21.1 idna 3.6 imagesize 1.4.1 importlib_metadata 7.0.1 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.3 markdown-it-py 3.0.0 MarkupSafe 2.1.3 mdit-py-plugins 0.4.0 mdurl 0.1.2 myst-parser 2.0.0 packaging 24.0 pathspec 0.12.1 platformdirs 4.2.0 pluggy 1.4.0 Pygments 2.17.2 pyproject-api 1.6.1 pyproject_hooks 1.0.0 pytest 8.1.1 python-dateutil 2.9.0.post0 PyYAML 6.0.1 requests 2.31.0 snowballstemmer 2.2.0 Sphinx 7.2.6 sphinx-tabs 3.4.5 sphinxcontrib-applehelp 1.0.8 sphinxcontrib-devhelp 1.0.5 sphinxcontrib-htmlhelp 2.0.5 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.7 sphinxcontrib-serializinghtml 1.1.10 tokenize_rt 5.2.0 tomli 2.0.1 tox 4.14.2 trove-classifiers 2024.3.13 urllib3 1.26.18 virtualenv 20.25.0 wheel 0.43.0 zipp 3.17.0 ```

henryiii commented 5 months ago

We really need to find and fix that logging issue! So much useless output. Anyway, looks like at least one case is looking for conda and not finding it. That's probably easily fixable. Edit: actually, it's already fixed.

henryiii commented 5 months ago

All three of the other failures are due to python -m venv failing. For example /usr/bin/python3 -m venv /tmp/pytest-of-tkloczko/pytest-1097/test_create_venv_backend0/venv fails. Could you try that by hand?

henryiii commented 5 months ago

Do you have python3-venv?

kloczek commented 4 months ago

Do you have python3-venv?

I'm running all my builds only in python 3 envs.

Just tested new 2024.04.15 and looks like it some progress. With python 3.10.14 and pytest 8.1.1 test suite fails now in 3 units

================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_command.py:383: Only run this test on Windows.
SKIPPED [1] tests/test_command.py:405: Only run this test on Windows.
SKIPPED [1] tests/test_sessions.py:1037: Missing conda command.
SKIPPED [1] tests/test_tox_to_nox.py:284: Not supported in tox 4.
SKIPPED [1] tests/test_virtualenv.py:156: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:185: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:197: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:211: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:241: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:247: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:264: Missing uv command.
SKIPPED [1] tests/test_virtualenv.py:431: Missing uv command.
SKIPPED [1] tests/test_virtualenv.py:463: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:476: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:488: Missing conda command.
SKIPPED [4] tests/test_virtualenv.py:507: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:532: Missing uv command.
SKIPPED [1] tests/test_virtualenv.py:532: Missing conda command.
SKIPPED [1] tests/test_virtualenv.py:561: Missing uv command.
SKIPPED [1] tests/test_virtualenv.py:587: Missing uv command.
SKIPPED [1] tests/test_virtualenv.py:687: Python 2.7 unsupported for virtualenv>=20.22.0
XPASS tests/test_logger.py::test_no_color_timestamp[color]
FAILED tests/test_virtualenv.py::test_stale_environment[venv-virtualenv-True] - nox.command.CommandFailed: Returned code 1
FAILED tests/test_virtualenv.py::test_create_reuse_venv_environment - nox.command.CommandFailed: Returned code 1
FAILED tests/test_virtualenv.py::test_create_venv_backend - nox.command.CommandFailed: Returned code 1
=================================================================== 3 failed, 540 passed, 24 skipped, 1 xpassed in 16.25s ===================================================================

O first look it looks like in at least some cases if conda is not installed some units are not skipped like it is other units.

Full log in attachment python-nox.FAIL.txt

henryiii commented 4 months ago

It still looks like python -m venv is failing; those three tests check the venv backend (which is only an opt-in backend and never actually required because we have virutalenv as a dependency). test_stale_environment[venv-virtualenv-True], test_create_reuse_venv_environment, and test_create_venv_backend all make venvs with venv; the rest of the tests use virtualenv.

I'm running all my builds only in python 3 envs.

Yes, but some package managers strip out the venv standard library module and require a separate package to use it, like python3-venv. Does /usr/bin/python3 -m venv /tmp/pytest-of-tkloczko/pytest-1097/test_create_venv_backend0/venv work? That's the command it is trying to run and is failing.

kloczek commented 4 months ago

/usr/bin/python3 -m venv /tmp/pytest-of-tkloczko/pytest-1097/test_create_venv_backend0/venv work? That's the command it is trying to run and is failing.

Yep it works.

wntrblm / nox