search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.41k stars 475 forks source link

./configure --disable-doc #32759

Closed mkoeppe closed 2 years ago

mkoeppe commented 3 years ago

This switch will disable the docbuild and avoid installing its many dependencies (sphinx...).

In contrast to other packages, pplpy's docbuild is enabled unless explicitly disabled by setting SAGE_SPKG_INSTALL_DOCS=no. We move it to a separate package pplpy_doc and make it a dependency of sagemath_doc_html.

Split out from #31396.

We also add argon2_cffi and nbclient to the packages disabled upon --disable-notebook.

Depends on #29039

CC: @tobiasdiez @dimpase @jhpalmieri @sagetrac-tmonteil

Component: build: configure

Author: Matthias Koeppe, John Palmieri

Branch/Commit: 43221d0

Reviewer: John Palmieri, Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/32759

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 51f3002 to b9da161

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b9da161Merge tag '9.5.beta6' into t/32759/__configure___disable_doc
jhpalmieri commented 2 years ago
comment:81

With ./configure --disable-docs && make, I'm getting

[pplpy-0.8.6] Finished installing pplpy-0.8.6
make --no-print-directory pplpy_doc-SAGE_DOCS-no-deps
Error: The installation tree SAGE_DOCS has been disabled
This Sage build is configured with configure --disable-doc, so building the documentation will not work.
make[4]: *** [pplpy_doc-SAGE_DOCS-no-deps] Error 1
make[3]: *** [/var/lib/sage/installed/pplpy_doc-0.8.6] Error 2
make[3]: *** Waiting for unfinished jobs....

This even happens after I made this change:

diff --git a/configure.ac b/configure.ac
index cec63aa918..fab7171839 100644
--- a/configure.ac
+++ b/configure.ac
@@ -475,7 +475,7 @@ AC_ARG_ENABLE([doc],
   AS_HELP_STRING([--disable-doc],
                  [disable build of the Sage documentation and packages depending on it]), [
     dnl Disable packages needed for docbuilding
-    for pkg in sage_docbuild alabaster babel snowballstemmer idna urllib3 certifi charset_normalizer requests imagesize sphinx sphinxcontrib_devhelp sphinxcontrib_jsmath sphinxcontrib_serializinghtml sphinxcontrib_applehelp sphinxcontrib_htmlhelp sphinxcontrib_qthelp sphinxcontrib_websupport; do
+    for pkg in sage_docbuild alabaster babel snowballstemmer idna urllib3 certifi charset_normalizer requests imagesize sphinx sphinxcontrib_devhelp sphinxcontrib_jsmath sphinxcontrib_serializinghtml sphinxcontrib_applehelp sphinxcontrib_htmlhelp sphinxcontrib_qthelp sphinxcontrib_websupport pplpy_doc; do
       AS_VAR_SET([SAGE_ENABLE_$pkg], [$enableval])
     done
     dnl Disable the docbuild by disabling the install tree for documentation
jhpalmieri commented 2 years ago
comment:82

It seems to be okay now. Maybe I need to run ./bootstrap or something similar (although the old config.log said that it wasn't building pplpy_doc.)

mkoeppe commented 2 years ago
comment:83

Yes, it sounds like bootstrap wasn't run.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

d626aa8build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no
82ce539configure.ac: If --disable-notebook and --disable-sagelib, do not build zeromq etc.
02505capkgs/sage-conf_pypi/MANIFEST.in: Update from #31396
aeca1a0Makefile (pypi-sdists): Add sage_setup
cd3dce9pkgs/sage-conf_pypi/setup.py: Update directory of configured sage_conf.py
c695921pkgs/sage-conf/sage_conf.py.in (SAGE_SPKG_WHEELS): Update so it works if SAGE_VENV != SAGE_LOCAL
cebcda3Merge tag '9.5.beta4' into t/29039/pip_installable_package_sage_bootstrap
0026892Merge tag '9.5.beta5' into t/29039/pip_installable_package_sage_bootstrap
b5cf9f6Merge #29039
04df2b7build/make/Makefile.in (all-start): Use all-sage-docs, not doc
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from b9da161 to 04df2b7

mkoeppe commented 2 years ago

Changed dependencies from #31356, #32745, #32836 to #31356, #32745, #32836, #29039

mkoeppe commented 2 years ago
comment:85

Merged #29039 to resolve a merge conflict

mkoeppe commented 2 years ago
comment:86

Possibly the last commit also fixed comment:81

jhpalmieri commented 2 years ago
comment:87

Everything builds for me, but my computer continues to misbehave, so I am not getting meaningful output on doctests. This is with OS X Monterey, and I have merged #32852. Many doctests involving plotting are timing out, so I get lots of doctest failures, and I don't know why. I tried a few sample plots interactively and they work.

mkoeppe commented 2 years ago
comment:88

Running some tests with EXTRA_CONFIGURE_ARGS="--disable-doc" tox -e docker-ubuntu-focal-standard.

Build went through but I am getting these failures on make ptestlong:

sage -t --long --random-seed=28205743117922504880455715083835945768 src/doc/en/tutorial/conf.py
    ModuleNotFoundError in doctesting framework
**********************************************************************
Traceback (most recent call last):
  File "/sage/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/sage/doctest/forker.py", line 2465, in __call__
    doctests, extras = self._run(runner, options, results)
  File "/sage/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/sage/doctest/forker.py", line 2513, in _run
    doctests, extras = self.source.create_doctests(sage_namespace)
  File "/sage/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/sage/doctest/sources.py", line 733, in create_doctests
    load(filename, namespace) # errors raised here will be caught in DocTestTask
  File "/sage/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/sage/repl/load.py", line 252, in load
    exec(code, globals)
  File "./conf.py", line 12, in <module>
  File "/sage/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/sage/docs/conf.py", line 3, in <module>
    import sphinx
ModuleNotFoundError: No module named 'sphinx'

File "src/doc/en/thematic_tutorials/sandpile.rst", line 1457, in doc.en.thematic_tutorials.sandpile
Failed example:
    Sandpile.help()
[formatting details]
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 04df2b7 to 7e980b1

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

7e980b1src/doc: Mark conf.py files # nodoctest
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

4fd9a83src/doc/en/thematic_tutorials/sandpile.rst: Do not include the full output of Sandpile.help() in the doctest
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 7e980b1 to 4fd9a83

mkoeppe commented 2 years ago
comment:91

A few more failures

sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/databases/findstat.py  # 94 doctests failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/doctest/test.py  # 7 doctests failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/doctest/sources.py  # 1 doctest failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/misc/bindable_class.py  # 2 doctests failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/misc/sagedoc.py  # 1 doctest failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/misc/sphinxify.py  # 7 doctests failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/misc/sageinspect.py  # 1 doctest failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/repl/ipython_tests.py  # 2 doctests failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/sandpiles/sandpile.py  # 1 doctest failed
sage -t --long --random-seed=184472493071993940636041244744568712523 src/sage/tests/lazy_imports.py  # 1 doctest failed
mkoeppe commented 2 years ago
comment:92

The findstat failures are:

      File "/sage/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/sage/databases/findstat.py", line 240, in <module>
        import requests
    ModuleNotFoundError: No module named 'requests'
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b0e1612src/setup.cfg.m4, src/requirements.txt.m4, build/pkgs/sagelib/dependencies: Add requests
f6ba517configure.ac (--disable-doc): Do not disable requests and its dependencies, needed for sage.databases.findstat
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 4fd9a83 to f6ba517

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

be64b25src/sage/sandpiles/sandpile.py: Make doctest pass even if sphinx formatting is not available
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from f6ba517 to be64b25

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

e4cb681src/sage/misc/sagedoc.py: Fix doctest
be532a2src/sage/misc/sphinxify.py: Use module-level # sage.doctest: optional - sphinx
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from be64b25 to be532a2

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from be532a2 to 6f8f8e0

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

86db757src/sage/misc/bindable_class.py: In doctests, strip output of sage_getdoc
0df932dsrc/sage/misc/sageinspect.py: Make doctest pass even if sphinx formatting is not available
6f8f8e0src/sage/repl/ipython_tests.py: Make doctest pass even if sphinx formatting is not available
jhpalmieri commented 2 years ago
comment:97

On an Ubuntu virtual machine, I'm getting doctest failures (with ./configure --disable-doc) because the alabaster package is not installed:

File "src/sage/misc/package.py", line 388, in sage.misc.package.installed_packages
Failed example:
    sorted(installed_packages().keys())  # optional - build
Expected:
    [...'alabaster', ...'sage_conf', ...]
Got:
    ['appnope',
     'arb',
     'argcomplete',
     'argon2_cffi',
     'attrs',
     'backcall',
     'bleach',
     'cddlib',
     'certifi',
     'cffi',
     'charset_normalizer',
     'combinatorial_designs',
     'conway_polynomials',
     'cvxopt',
     'cycler',
     'cypari',
     'cysignals',
     'cython',
     'dateutil',
     'debugpy',
     'decorator',
     'defusedxml',
     'docutils',
     'ecl',
     'eclib',
     'elliptic_curves',
     'entrypoints',
     'flint',
     'flit_core',
     'fplll',
     'fpylll',
     'gap',
     'gengetopt',
     'giac',
     'gmpy2',
     'graphs',
     'html5lib',
     'idna',
     'importlib_metadata',
     'importlib_resources',
     'ipykernel',
     'ipython',
     'ipython_genutils',
     'ipywidgets',
     'jedi',
     'jinja2',
     'jmol',
     'jsonschema',
     'jupyter_client',
     'jupyter_core',
     'jupyter_jsmol',
     'jupyterlab_pygments',
     'kiwisolver',
     'lcalc',
     'linbox',
     'markupsafe',
     'mathjax',
     'matplotlib',
     'matplotlib_inline',
     'maxima',
     'memory_allocator',
     'mistune',
     'mpmath',
     'nbclient',
     'nbconvert',
     'nbformat',
     'nest_asyncio',
     'networkx',
     'notebook',
     'numpy',
     'packaging',
     'pandocfilters',
     'pari',
     'pari_galdata',
     'pari_seadata_small',
     'parso',
     'pexpect',
     'pickleshare',
     'pillow',
     'pip',
     'pkgconfig',
     'pluggy',
     'polytopes_db',
     'pplpy',
     'prometheus_client',
     'prompt_toolkit',
     'ptyprocess',
     'py',
     'pybind11',
     'pycparser',
     'pycygwin',
     'pygments',
     'pyparsing',
     'pyrsistent',
     'pytz',
     'pyzmq',
     'ratpoints',
     'requests',
     'rpy2',
     'sage_conf',
     'sage_setup',
     'sagelib',
     'sagenb_export',
     'sagetex',
     'scipy',
     'send2trash',
     'setuptools',
     'setuptools_scm',
     'setuptools_wheel',
     'simplegeneric',
     'singular',
     'six',
     'sympy',
     'terminado',
     'testpath',
     'thebe',
     'threejs',
     'tomli',
     'tornado',
     'traitlets',
     'typing_extensions',
     'tzlocal',
     'urllib3',
     'vcversioner',
     'wcwidth',
     'webencodings',
     'wheel',
     'widgetsnbextension',
     'zipp']
**********************************************************************
File "src/sage/misc/package.py", line 390, in sage.misc.package.installed_packages
Failed example:
    installed_packages()['alabaster']  # optional - build, random
Exception raised:
    Traceback (most recent call last):
      File "/home/john/Sage/sage-9.5.beta5/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/john/Sage/sage-9.5.beta5/local/var/lib/sage/venv-python3.8/lib/python3.8/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.installed_packages[1]>", line 1, in <module>
        installed_packages()['alabaster']  # optional - build, random
    KeyError: 'alabaster'
*********************************************************************
jhpalmieri commented 2 years ago
comment:98

Also, and maybe this is just how it will have to be, if I use ./configure --disable-doc and then make and then make doc, it installs sphinx and various other packages, prints a good error message when trying to install pplpy_doc, but of continues to install whatever it's in the middle of, so the error message gets buried.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

1807446src/sage/misc/package.py: Use gmpy2 instead of alabaster as example in doctest
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 6f8f8e0 to 1807446

mkoeppe commented 2 years ago
comment:100

Replying to @jhpalmieri:

Also, and maybe this is just how it will have to be, if I use ./configure --disable-doc and then make and then make doc, it installs sphinx and various other packages, prints a good error message when trying to install pplpy_doc, but of continues to install whatever it's in the middle of, so the error message gets buried.

I think it would be too complicated to check for disable-doc earlier

mkoeppe commented 2 years ago
comment:101

Replying to @jhpalmieri:

On an Ubuntu virtual machine, I'm getting doctest failures (with ./configure --disable-doc) because the alabaster package is not installed

This was also the last failure on my test with macOS, fixed in 1807446 above.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 1807446 to f4aaa81

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

f4aaa81README.md, build/make/install: Update - documentation no longer has special log file location
jhpalmieri commented 2 years ago
comment:103

make doc now builds both html and pdf, whereas before it just built html. How many people will this surprise?

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from f4aaa81 to 651c986

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

651c986build/make/Makefile.in (doc): Revert to just doc-html (not both doc-html and doc-pdf)
mkoeppe commented 2 years ago
comment:105

Thanks for catching this; an unnecessary change

mkoeppe commented 2 years ago

Description changed:

--- 
+++ 
@@ -1,7 +1,6 @@
 This switch will disable the docbuild and installation of its many dependencies (sphinx...)

-In contrast to other packages, `ppl`'s docbuild is enabled unless explicitly disabled by setting `SAGE_SPKG_INSTALL_DOCS=no`.
-We conditionalize the dependency of `ppl` on this and also on `sphinx` being in the list of installed packages.
+In contrast to other packages, `pplpy`'s docbuild is enabled unless explicitly disabled by setting `SAGE_SPKG_INSTALL_DOCS=no`. We move it to a separate package `pplpy_doc` and make it a dependency of `sagemath_doc_html`.

 Split out from #31396.
jhpalmieri commented 2 years ago
comment:107

This part obviously works, but let me see if I understand the syntax:

$(1)-$(4)-no-deps:
    +@if [ -z '$$($(4))' ]; then \
        echo "Error: The installation tree $(4) has been disabled" 2>&1; \
        echo "$$($(4)_DISABLED_MESSAGE)" 2>&1; \
        exit 1; \

Is the following right? In the case of this ticket, $(4) is SAGE_DOCS, and then $$($(4)) evaluates the Makefile variable SAGE_DOCS, which will be empty if docs have been disabled.

mkoeppe commented 2 years ago
comment:108

That's right.

jhpalmieri commented 2 years ago
comment:109

Okay, let's merge it.

jhpalmieri commented 2 years ago

Reviewer: John Palmieri, Matthias Koeppe

mkoeppe commented 2 years ago
comment:110

Thank you!

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from 651c986 to edaa3e6

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

edaa3e6Merge tag '9.5.beta8' into t/32759/__configure___disable_doc
mkoeppe commented 2 years ago

Changed dependencies from #31356, #32745, #32836, #29039 to #29039

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Changed commit from edaa3e6 to 2dd7cf5

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 2 years ago

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

2a4a323Merge tag '9.5.beta7' into t/29039/pip_installable_package_sage_bootstrap
a7fa5aaMerge #29039
2dd7cf5Merge tag '9.5.beta9' into t/32759/__configure___disable_doc
mkoeppe commented 2 years ago
comment:114

Merged latest beta to resolve merge conflicts.