sage -t --installed

[mkoeppe]

This new option is similar to --all but

• it tests all Sage modules installed (in SAGE_LIB) instead of SAGE_SRC. (This will be useful for testing modularized distributions such as #32601.)
• it tests the installed Sage documentation (specifically, the .rst.txt files installed in html/**/_sources) instead of SAGE_DOC_SRC.

$ ./sage -tp --installed
too many failed tests, not using stored timings
Running doctests with ID 2022-02-23-21-52-47-11831639.
Using --optional=ccache,cryptominisat,debugpy,e_antic,homebrew,igraph,jupymake,lrslib,normaliz,pip,polytopes_db_4d,pycryptosat,pynormaliz,sage,sage_spkg
Features to be detected: [...] 
Doctesting all installed modules of the Sage library.
Sorting sources by runtime so that slower doctests are run first....
Doctesting 3522 files using 8 threads.
sage -t --random-seed=334799075370772165291245032923424842623 local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/groups/matrix_gps/heisenberg.py
    [32 tests, 1.83 s]
sage -t --random-seed=334799075370772165291245032923424842623 local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/combinat/crystals/alcove_path.py
    [293 tests, 4.16 s]
...

(Actually, because portions of namespace packages can be installed in separate locations, the variable sage.env.SAGE_LIB is no longer meaningful. Instead, we make use of the __path__ attribute of packages, https://python.readthedocs.io/en/stable/reference/import.html#module-path, which in the case of namespace packages is an iterable.)

This would also make sense for distribution packaging, which no longer would have to rely on the fallbacks SAGE_SRC -> SAGE_LIB and SAGE_DOC_SRC -> SAGE_DOC.

Part of Meta-ticket #33037.

CC: @kiwifb @antonio-rojas @tobiasdiez

Component: doctest framework

Author: Matthias Koeppe

Branch/Commit: b4d2b8e

Reviewer: François Bissey, Antonio Rojas

Issue created by migration from https://trac.sagemath.org/ticket/33407

[kiwifb]

comment:42

I have now run a full doctest on sage-on-gentoo with --installed. Because we have not acted yet on SAGE_SRC/SAGE_LIB/SAGE_DOC_SRC checking with --all was just business as usual. But nothing else came up (apart from the current lot of failures in #33141 and new giac issues that I have to inspect more closely).

[kiwifb]

comment:43

While it is unrelated to eliminating SAGE_* variables, there is something related to testing an installed sage that we need to address in a follow up. pytest doesn't work if you cannot write in the "rootdir" of what is tested.

There are two issues with that. First it may leave unwanted files all over the place. Second it fails miserably if you try to test something that is read-only.

============================= test session starts ==============================
platform linux -- Python 3.10.2, pytest-7.0.1, pluggy-1.0.0
rootdir: /usr
plugins: hypothesis-6.38.0
collected 0 items

=============================== warnings summary ===============================
../../usr/lib/python3.10/site-packages/_pytest/cacheprovider.py:433
  /usr/lib/python3.10/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /usr/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../usr/lib/python3.10/site-packages/_pytest/stepwise.py:52
  /usr/lib/python3.10/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /usr/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

There seem to be some option to change where the cache is created (-o cache_dir=$somelocation) but I have so far failed to integrate it successfully in sage-runtest when pytest is called.

[kiwifb]

comment:45

Follow up for pytest at #33521.

[antonio-rojas]

Changed reviewer from François Bissey to François Bissey, Antonio Rojas

[antonio-rojas]

comment:46

Working fine here.

[mkoeppe]

comment:47

Thanks all for the review!

[vbraun]

Changed branch from u/mkoeppe/sage__t___all_installed to b4d2b8e