generate-ci'd emscripten pytest fails before reaching module code

[chrysn]

Bug Description

In cbor-diag-py, which is an almost trivial[^1] project following the maturing scaffolding of maturin new and generate-ci, the emscripten tests fail, seemingly before they even touch code.

Excerpt of [the latest CI run](https://github.com/chrysn/cbor-diag-py/actions/runs/8961096846/job/24608342117)

``` [pytest] Run set -e xbuild environment already exists, skipping download Installing xbuild environment Creating Pyodide virtualenv at .venv ... Configuring virtualenv ... Installing standard library Successfully created Pyodide virtual environment! Looking in indexes: https://pypi.org/simple, file:///home/runner/work/cbor-diag-py/cbor-diag-py/.pyodide-xbuildenv/xbuildenv/pyodide-root/pypa_index Looking in links: dist Collecting cbor-diag Downloading https://cdn.jsdelivr.net/pyodide/v0.25.1/full/cbor_diag-1.0.1-cp311-cp311-emscripten_3_1_46_wasm32.whl (362 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 362.9/362.9 kB 14.3 MB/s eta 0:00:00 Installing collected packages: cbor-diag Successfully installed cbor-diag-1.0.1 WARNING: Location 'file:/home/runner/work/cbor-diag-py/cbor-diag-py/.pyodide-xbuildenv/xbuildenv/pyodide-root/pypa_index/pip/' is ignored: it is neither a file nor a directory. Looking in indexes: https://pypi.org/simple, file:///home/runner/work/cbor-diag-py/cbor-diag-py/.pyodide-xbuildenv/xbuildenv/pyodide-root/pypa_index Collecting pytest Downloading pytest-8.2.0-py3-none-any.whl.metadata (7.5 kB) Collecting iniconfig (from pytest) Downloading https://cdn.jsdelivr.net/pyodide/v0.25.1/full/iniconfig-2.0.0-py3-none-any.whl (14 kB) Requirement already satisfied: packaging in ./.venv/lib/python3.11/site-packages (from pytest) (23.1) Collecting pluggy<2.0,>=1.5 (from pytest) Downloading pluggy-1.5.0-py3-none-any.whl.metadata (4.8 kB) Downloading pytest-8.2.0-py3-none-any.whl (339 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 339.2/339.2 kB 8.8 MB/s eta 0:00:00 Downloading pluggy-1.5.0-py3-none-any.whl (20 kB) Installing collected packages: pluggy, iniconfig, pytest Successfully installed iniconfig-2.0.0 pluggy-1.5.0 pytest-8.2.0 ============================= test session starts ============================== platform emscripten -- Python 3.11.3, pytest-8.2.0, pluggy-1.5.0 rootdir: /home/runner/work/cbor-diag-py/cbor-diag-py configfile: pytest.ini collected 1 item / 10 errors ==================================== ERRORS ==================================== _ ERROR collecting emsdk-cache/emsdk-main/upstream/emscripten/test/test_benchmark.py _ ImportError while importing test module '/home/runner/work/cbor-diag-py/cbor-diag-py/emsdk-cache/emsdk-main/upstream/emscripten/test/test_benchmark.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /lib/python311.zip/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) emsdk-cache/emsdk-main/upstream/emscripten/test/test_benchmark.py:20: in import clang_native emsdk-cache/emsdk-main/upstream/emscripten/test/clang_native.py:10: in from tools.shared import PIPE, run_process, CLANG_CC, CLANG_CXX E ModuleNotFoundError: No module named 'tools' _ ERROR collecting emsdk-cache/emsdk-main/upstream/emscripten/test/test_browser.py _ ImportError while importing test module '/home/runner/work/cbor-diag-py/cbor-diag-py/emsdk-cache/emsdk-main/upstream/emscripten/test/test_browser.py'. Hint: make sure your test modules/packages have valid Python names. ```

Given that nothing in the errors appear to be related to the cbor-diag module, it appears that this is an issue with maturin or the actions added in generate-ci. I'm a bit lost at why the emscripten tests are even called, do you have any pointers to track this down further?

[^1]: The package uses altered paths and a manual init to avoid the public modulename.modulename API guarantee tracked in #1399

Your maturin version (maturin --version)

1.5.1

Your Python version (python -V)

3.11.8

Your pip version (pip -V)

24.0

What bindings you're using

pyo3

Does cargo build work?

• [X] Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

• [ ] Yes

Steps to Reproduce

The issue is with what happens on GitHub actions, I don't think this can be reproduced locally.

[messense]

I don't know either, @hoodmane any idea?

[hoodmane]

cc @ryanking13.

I'm not sure why it's trying to run those tests but I think they were meant to run in the emscripten source tree and weren't meant to run on install. I wonder why they are running here.

[ryanking13]

It looks like emscripten is installed under emsdk-cache directory (the directory specified in actions-cache-folder option in the setup-emsdk action). And emscripten contains a bunch of Python test files in their repository, so pytest added them to the test suite.

So I guess you need to

1. Install emscripten somewhere out of current working directory, or
2. Specify the test file paths clearly when running pytest

[chrysn]

Just to not drop the ball, is that something I can act on in my repository (doesn't seem like it, it doesn't touch emsdk installation), something where we'd need to change the generate output, or something where we'd need to pull in maintainers of the action repositories supplying the CI steps?

[ryanking13]

is that something I can act on in my repository

Well, I think the easiest way, and what I would like to recommend, is specifying the files/directory to test by pytest clearly. By adding testpaths in pytest.ini, so that even if some files are unintentionally created in the local directory during CI, pytest will only put the specified directory in the test scope.