search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.45k stars 481 forks source link

pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf #29868

Open mkoeppe opened 4 years ago

mkoeppe commented 4 years ago

Similar to #29411 + #29950, which made sagelib a pip-installable distribution (which is then installed by the sage-the-distribution script package sagelib), we change the build/installation scheme of the sage documentation.

We create pip-installable packages sagemath-doc-html and sagemath-doc-pdf. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.

We create a package sagemath-doc-inventory, which is in charge of building and installing the inventory files. It is a build dependency of sagemath-doc-html and sagemath-doc-pdf.

All of these packages depend on sagemath-doc-src. It has a build-system dependency on sagelib. An sdist of it consists of SAGE_SRC/doc. A wheel of sagemath-doc-src (and thus an installation) contains a copy of the sources (including the auto-generated rst files) in share/doc/sage/src/ and the doctrees in share/doc/sage/doctrees. The incremental build features of the docbuild will only be active when the package sagemath-doc-src is installed directly using setup.py install, setup.py develop or pip install --editable. In these cases, the doctrees (including environment pickes) are kept in the source directory. pip wheel will always make a from-scratch build of the documentation, eliminating incremental build errors.

The script sage-grepdoc will be provided by sagemath-doc-html.

Follow-up tickets:

See also:

Depends on #33852

CC: @isuruf @kiwifb @antonio-rojas @timokau @nbruin @dimpase @jhpalmieri @mwageringel @zlscherr

Component: build

Keywords: sd111

Issue created by migration from https://trac.sagemath.org/ticket/29868

mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,3 +1,5 @@
 Similar to #29411, which makes sagelib a script package, we make the sage docbuild a script package.

+See also: #30010 - Split `sage_setup.docbuild` out to a separate package
+
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,5 +1,8 @@
-Similar to #29411, which makes sagelib a script package, we make the sage docbuild a script package.
+Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the installation scheme of the sage documentation.

+We create pip-installable packages `sage-doc-html` and `sage-doc-pdf`. 
+
+When installed using pip, they provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).

 See also: #30010 - Split `sage_setup.docbuild` out to a separate package
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -4,5 +4,7 @@

 When installed using pip, they provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).

+Also, the script `sage-grepdoc` will be provided by `sage-doc-html`.
+
 See also: #30010 - Split `sage_setup.docbuild` out to a separate package
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -6,5 +6,7 @@

 Also, the script `sage-grepdoc` will be provided by `sage-doc-html`.

-See also: #30010 - Split `sage_setup.docbuild` out to a separate package
+See also: 
+- #30010 - Split `sage_setup.docbuild` out to a separate package
+- #29097 - `build/make/Makefile.in`: Rename make targets `SPKG-clean` to `SPKG-uninstall`
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,8 +1,8 @@
 Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the installation scheme of the sage documentation.

-We create pip-installable packages `sage-doc-html` and `sage-doc-pdf`. 
+We create pip-installable packages `sage-doc-html` and `sage-doc-pdf`. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.

-When installed using pip, they provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
+When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).

 Also, the script `sage-grepdoc` will be provided by `sage-doc-html`.
mkoeppe commented 3 years ago

Changed keywords from none to sd111

mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -1,10 +1,10 @@
 Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the installation scheme of the sage documentation.

-We create pip-installable packages `sage-doc-html` and `sage-doc-pdf`. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.
+We create pip-installable packages `sagemath-doc-html` and `sagemath-doc-pdf`. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.

 When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).

-Also, the script `sage-grepdoc` will be provided by `sage-doc-html`.
+Also, the script `sage-grepdoc` will be provided by `sagemath-doc-html`.

 See also: 
 - #30010 - Split `sage_setup.docbuild` out to a separate package
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -9,4 +9,5 @@
 See also: 
 - #30010 - Split `sage_setup.docbuild` out to a separate package
 - #29097 - `build/make/Makefile.in`: Rename make targets `SPKG-clean` to `SPKG-uninstall`
+- #29231 (add intersphinx mapping for SciPy) - with discussion about docbuild and inventory files
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -1,10 +1,18 @@
-Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the installation scheme of the sage documentation.
+Similar to #29411 + #29950, which made `sagelib` a pip-installable distribution (which is then installed by the sage-the-distribution script package `sagelib`), we change the build/installation scheme of the sage documentation.

 We create pip-installable packages `sagemath-doc-html` and `sagemath-doc-pdf`. By providing wheels on pypi, users can get access to an offline copy of the documentation without having to build it themselves.

-When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
+We create a package `sagemath-doc-inventory`, which is in charge of building and installing the inventory files.
+It is a build dependency of `sagemath-doc-html` and `sagemath-doc-pdf`.

-Also, the script `sage-grepdoc` will be provided by `sagemath-doc-html`.
+We no longer install `doctrees` in `$SAGE_LOCAL/share/doc/sage/`.
+The incremental build features of the docbuild will only be active when the package is installed directly using `setup.py install`, `setup.py develop` or `pip install --editable`.  In these cases, the doctrees (including environment pickes) are kept in the source directory.
+`pip wheel` will always make a from-scratch build of the documentation, eliminating incremental build errors.
+
+The script `sage-grepdoc` will be provided by `sagemath-doc-html`.
+
+Follow-up tickets:
+- When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).

 See also: 
 - #30010 - Split `sage_setup.docbuild` out to a separate package
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -5,8 +5,8 @@
 We create a package `sagemath-doc-inventory`, which is in charge of building and installing the inventory files.
 It is a build dependency of `sagemath-doc-html` and `sagemath-doc-pdf`.

-We no longer install `doctrees` in `$SAGE_LOCAL/share/doc/sage/`.
-The incremental build features of the docbuild will only be active when the package is installed directly using `setup.py install`, `setup.py develop` or `pip install --editable`.  In these cases, the doctrees (including environment pickes) are kept in the source directory.
+All of these packages depend on `sagemath-doc-src`. It has a build-system dependency on sagelib. An `sdist` of it consists of `SAGE_SRC/doc`. A `wheel` of `sagemath-doc-src` (and thus an installation) contains a copy of the sources (including the auto-generated rst files) in `share/doc/sage/src/` and the doctrees in `share/doc/sage/doctrees`.
+The incremental build features of the docbuild will only be active when the package `sagemath-doc-src` is installed directly using `setup.py install`, `setup.py develop` or `pip install --editable`.  In these cases, the doctrees (including environment pickes) are kept in the source directory.
 `pip wheel` will always make a from-scratch build of the documentation, eliminating incremental build errors.

 The script `sage-grepdoc` will be provided by `sagemath-doc-html`.
mkoeppe commented 3 years ago

Dependencies: #31356

mkoeppe commented 3 years ago
comment:14

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

mkoeppe commented 3 years ago
comment:15

Input and help from docbuild experts is very welcome.

mkoeppe commented 3 years ago
comment:16

Some questions:

kiwifb commented 3 years ago
comment:17

Replying to @mkoeppe:

Some questions:

  • Is there a way to only generate the doctrees but not build HTML or PDF?

I do believe there is a sphinx option for doing just that.

  • Do the HTML and PDF builds need to be able to run the Sage library, or can they build these formats if given only the doctrees?

As far as I know they need to be able to run the sage library. In particular all graphs/plots are generated during the documentation build.

kiwifb commented 3 years ago
comment:18

Replying to @kiwifb:

Replying to @mkoeppe:

Some questions:

  • Is there a way to only generate the doctrees but not build HTML or PDF?

I do believe there is a sphinx option for doing just that.

I cannot find such an option anymore. If it existed. There is an option to say where the doctrees should be stored and found, but nothing on producing it in isolation.

jhpalmieri commented 3 years ago
comment:19

Well, ./sage --docbuild all inventory builds the inventory files and the doctree files, but not html or pdf.

jhpalmieri commented 3 years ago
comment:20

Replying to @mkoeppe:

Some questions:

  • Who generates files such as src/doc/en/reference/monoids/sage/monoids/free_monoid.rst, and should they be part of what sagemath-doc-src installs (and thus ships with the wheel)?

The autodoc procedure does this, with sage/monoids/free_monoid.py as input. (Edit: or maybe just src/doc/en/reference/monoids/index.rst as input?) I'm not sure precisely how this is done, but the file sage_docbuild/ext/sage_autodoc.py is certainly relevant. (This is Sage's fork of the Sphinx autodoc module.)

jhpalmieri commented 3 years ago
comment:21

No, I'm wrong about that. It's done in sage_docbuild/__init__.py, in the write_auto_rest_file method.

mkoeppe commented 3 years ago

Changed dependencies from #31356 to #31356, #32713

mkoeppe commented 2 years ago
comment:25

According to #33064 comment:19, there is no need to install doctrees in SAGE_LOCAL, not even for introspection (sage.misc.sphinxify)

slel commented 2 years ago

Description changed:

--- 
+++ 
@@ -15,7 +15,7 @@
 - When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).

 See also: 
-- #30010 - Split `sage_setup.docbuild` out to a separate package
-- #29097 - `build/make/Makefile.in`: Rename make targets `SPKG-clean` to `SPKG-uninstall`
-- #29231 (add intersphinx mapping for SciPy) - with discussion about docbuild and inventory files
+- #30010: Split `sage_setup.docbuild` out to a separate package
+- #29097: `build/make/Makefile.in`: Rename make targets `SPKG-clean` to `SPKG-uninstall`
+- #29231: Add intersphinx mapping for SciPy -- with discussion about docbuild and inventory files
mkoeppe commented 2 years ago

Changed dependencies from #31356, #32713 to #33852

mkoeppe commented 2 years ago

Description changed:

--- 
+++ 
@@ -13,6 +13,10 @@

 Follow-up tickets:
 - When installed using pip, the packages provide `nbextension`s that provide offline help in a Jupyter notebook. This is useful in particular with local notebooks connecting to remote sage kernels (see  #30306 (3)).
+   - https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Distributing%20Jupyter%20Extensions%20as%20Python%20Packages.html
+- Or alternatively as a JupyterLab extension
+   - https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html
+

 See also: 
 - #30010: Split `sage_setup.docbuild` out to a separate package
kwankyu commented 1 year ago
  • Who generates files such as src/doc/en/reference/monoids/sage/monoids/free_monoid.rst,

The autodoc procedure does this, with sage/monoids/free_monoid.py as input. (Edit: or maybe just src/doc/en/reference/monoids/index.rst as input?) I'm not sure precisely how this is done, but the file sage_docbuild/ext/sage_autodoc.py is certainly relevant. (This is Sage's fork of the Sphinx autodoc module.)

No. Reference doc builder (part of Sage, in src/sage_docbuild) is responsible for that, as tangentially explained in https://github.com/sagemath/sage/pull/36692#issuecomment-1806976665. Then autodoc (extension to Sphinx) is responsible to render the generated rst files. We are shipping sage-specific version of autodoc, but that should be replaced eventually with official autodoc provided by Sphinx

and should they be part of what sagemath-doc-src installs (and thus ships with the wheel)?

Yes, if it intends to include all rst files from which htmls (or pdfs) are generated by Sphinx.

kwankyu commented 1 year ago

comment:21 No, I'm wrong about that. It's done in sage_docbuild/__init__.py, in the write_auto_rest_file method.

Now the builders are in src/sage_docbuild/builders.py.