search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.28k stars 438 forks source link

Meta-ticket: Sage docbuild #20080

Open nthiery opened 8 years ago

nthiery commented 8 years ago

Docbuild tickets:

Sphinx tickets:

See also:

Overview of Sphinx custom code in Sage's documentation build system

The goal of this ticket is to write an overview of the various bits and pieces of custom Sphinx code in Sage, with the long term goal to see which pieces could be:

Here is a list of files with their relative roles:

src/sage/docs

src/sage_docbuild (was src/sage_setup/docbuild)

src/sage_docbuild/ext (was src/sage_setup/docbuild/ext)

src/sage/misc

CC: @egourgoulhon @jhpalmieri @dimpase @kwankyu @haraldschilly

Component: documentation

Issue created by migration from https://trac.sagemath.org/ticket/20080

nthiery commented 8 years ago

Description changed:

--- 
+++ 
@@ -5,8 +5,3 @@
 - cleaned up

 }}}
-
-
-
-
-}}}
nthiery commented 8 years ago

Description changed:

--- 
+++ 
@@ -1,7 +1,5 @@
-{{{!rst
 The goal of this ticket is to write an overview of the various bits and pieces of custom Sphinx code in Sage, with the long term goal to see which pieces could be:
 - thrown away, using features that are now in Sphinx
 - generalized and submitted upstream
 - cleaned up

-}}}
hivert commented 8 years ago

Description changed:

--- 
+++ 
@@ -3,3 +3,40 @@
 - generalized and submitted upstream
 - cleaned up

+Here is a lists of file in `src/doc/common` with their relative roles:
+
+- builder.py
+- build_options.py
+- conf.py
+- custom-sphinx-build.py
+- __init__.py :
+
+    empty file
+
+- inventory_builder.py:
+
+    A customized HTML builder which only generates intersphinx "object.inv"
+    inventory files and pickle files. The documentation files are not written.
+
+
+- multidocs.py:
+
+    This is a slightly hacked-up version of the Sphinx-multidoc plugin
+
+    The goal of this extension is to manage a multi documentation in Sphinx.
+    To be able to compile Sage's huge documentation in parallel, the
+    documentation is cut into a bunch of independent documentations called
+    "subdocs", which are compiled separately. There is a master document which
+    points to all the subdocs. The intersphinx extension ensures that the
+    cross-link between the subdocs are correctly resolved. However some work
+    is needed to build a global index. This is the goal of multidocs.
+
+    More precisely this extension ensures the correct merging of
+    - the todo list if this extension is activated;
+    - the python indexes;
+    - the list of python modules;
+    - the javascript index;
+    - the citations.
+
+- sage_autodoc.py
+
jdemeyer commented 8 years ago

Description changed:

--- 
+++ 
@@ -3,23 +3,28 @@
 - generalized and submitted upstream
 - cleaned up

-Here is a lists of file in `src/doc/common` with their relative roles:
+Here is a list of files in `src/doc/common` with their relative roles:

-- builder.py
-- build_options.py
-- conf.py
-- custom-sphinx-build.py
-- __init__.py :
+- `__init__.py`:

     empty file

-- inventory_builder.py:
+- `conf.py`
+
+Here is a list of files in `srcsrc/sage_setup/docbuild` with their relative roles:
+
+- `builder.py`
+
+- `build_options.py`
+
+- `custom-sphinx-build.py`
+
+- `ext/inventory_builder.py`:

     A customized HTML builder which only generates intersphinx "object.inv"
     inventory files and pickle files. The documentation files are not written.

-
-- multidocs.py:
+- `ext/multidocs.py`:

     This is a slightly hacked-up version of the Sphinx-multidoc plugin

@@ -38,5 +43,4 @@
     - the javascript index;
     - the citations.

-- sage_autodoc.py
-
+- `ext/sage_autodoc.py`
jdemeyer commented 8 years ago

Description changed:

--- 
+++ 
@@ -5,19 +5,19 @@

 Here is a list of files in `src/doc/common` with their relative roles:

-- `__init__.py`:
-
-    empty file
+- `__init__.py`: empty file

 - `conf.py`

 Here is a list of files in `srcsrc/sage_setup/docbuild` with their relative roles:

-- `builder.py`
+- `__init__.py`: top-level docbuilder, contains classes 
+
+- `__main__.py`: entry point for `sage --docbuild`, calls `main()` from `__init__.py`

 - `build_options.py`

-- `custom-sphinx-build.py`
+- `sphinxbuild.py`

 - `ext/inventory_builder.py`:
kiwifb commented 8 years ago
comment:6

I would like to mention sphinxify.py which is currently part of sagenb but should be moved to inside sage proper ASAP. sphinxify is called from sage/misc/sagedoc.py both at doc building time and run time.

jdemeyer commented 8 years ago

Description changed:

--- 
+++ 
@@ -3,13 +3,15 @@
 - generalized and submitted upstream
 - cleaned up

-Here is a list of files in `src/doc/common` with their relative roles:
+Here is a list of files with their relative roles:
+
+**src/doc/common**

 - `__init__.py`: empty file

 - `conf.py`

-Here is a list of files in `srcsrc/sage_setup/docbuild` with their relative roles:
+**src/sage_setup/docbuild**

 - `__init__.py`: top-level docbuilder, contains classes 

@@ -19,12 +21,14 @@

 - `sphinxbuild.py`

-- `ext/inventory_builder.py`:
+**src/sage_setup/docbuild/ext**
+
+- `inventory_builder.py`:

     A customized HTML builder which only generates intersphinx "object.inv"
     inventory files and pickle files. The documentation files are not written.

-- `ext/multidocs.py`:
+- `multidocs.py`:

     This is a slightly hacked-up version of the Sphinx-multidoc plugin

@@ -43,4 +47,8 @@
     - the javascript index;
     - the citations.

-- `ext/sage_autodoc.py`
+- `sage_autodoc.py`
+
+**sagenb/sagenb/misc**
+
+- `sphinxify.py`
jdemeyer commented 8 years ago

Description changed:

--- 
+++ 
@@ -13,7 +13,7 @@

 **src/sage_setup/docbuild**

-- `__init__.py`: top-level docbuilder, contains classes 
+- `__init__.py`: top-level docbuilder, contains classes for the various documents (e.g. reference manual).

 - `__main__.py`: entry point for `sage --docbuild`, calls `main()` from `__init__.py`
kcrisman commented 8 years ago
comment:10

I would like to mention sphinxify.py which is currently part of sagenb but should be moved to inside sage proper ASAP. sphinxify is called from sage/misc/sagedoc.py both at doc building time and run time.

I'm open to PRs, as this seems pretty reasonable - docbuild shouldn't depend on sagenb, though we would still want the doc to work from within it.

hivert commented 8 years ago

Description changed:

--- 
+++ 
@@ -9,17 +9,19 @@

 - `__init__.py`: empty file

-- `conf.py`
+- `conf.py`: standard Sphinx configuration file.

 **src/sage_setup/docbuild**

 - `__init__.py`: top-level docbuilder, contains classes for the various documents (e.g. reference manual).

-- `__main__.py`: entry point for `sage --docbuild`, calls `main()` from `__init__.py`
+- `__main__.py`: stub entry point for `sage --docbuild`, calls `main()` from `__init__.py`

-- `build_options.py`
+- `build_options.py`: Parse the option for building.

-- `sphinxbuild.py`
+- `sphinxbuild.py`: Sage's version of the sphinx-build script
+
+   mainly deals with logging and error reporting.

 **src/sage_setup/docbuild/ext**

@@ -27,6 +29,7 @@

     A customized HTML builder which only generates intersphinx "object.inv"
     inventory files and pickle files. The documentation files are not written.
+    This is supposed to be used with multidocs below.

 - `multidocs.py`:

@@ -41,14 +44,33 @@
     is needed to build a global index. This is the goal of multidocs.

     More precisely this extension ensures the correct merging of
-    - the todo list if this extension is activated;
-    - the python indexes;
-    - the list of python modules;
-    - the javascript index;
-    - the citations.
+
+        1. the todo list if this extension is activated;
+        2. the python indexes;
+        3. the list of python modules;
+        4. the javascript index;
+        5. the citations.

 - `sage_autodoc.py`
+
+    This is a patched version of sphinx.ext.autodoc, which started to diverge
+    before 2011. As far as I (Florent) understand, the role here is to
+
+        1. allows to correctly fetch the Cython doc and argspec
+        2. get correctly the argspec of callable (from the `_sage_argspec_`
+           attribute) #9976
+        3. deals correctly with LazyImport #17455
+        4. In particular document CachedFunction as a function and not a class
+           instance #9976.
+        5. correctly handles classe aliases and nested classed #7448 #5986
+
+    This file probably need to be completely reworked since Sphinx seems to
+    have now various plugin and Mixin.

 **sagenb/sagenb/misc**

 - `sphinxify.py`
+
+    script which calls Sphinx to format a single docstring for help from the
+    command line or the notebook. Has its own configuration.
+
embray commented 8 years ago
comment:12

Could someone clarify what

This is a slightly hacked-up version of the Sphinx-multidoc plugin

means?

I can't find any reference to a Sphinx multidoc plugin outside of Sage.

jdemeyer commented 8 years ago

Description changed:

--- 
+++ 
@@ -32,8 +32,6 @@
     This is supposed to be used with multidocs below.

 - `multidocs.py`:
-
-    This is a slightly hacked-up version of the Sphinx-multidoc plugin

     The goal of this extension is to manage a multi documentation in Sphinx.
     To be able to compile Sage's huge documentation in parallel, the
jdemeyer commented 8 years ago
comment:13

Fixed :-)

jdemeyer commented 8 years ago

Description changed:

--- 
+++ 
@@ -65,7 +65,7 @@
     This file probably need to be completely reworked since Sphinx seems to
     have now various plugin and Mixin.

-**sagenb/sagenb/misc**
+**src/sage/misc**

 - `sphinxify.py`
embray commented 8 years ago
comment:15

I'm having "fun" finally taking the time to fully understand the doc builds system... :)

dimpase commented 4 years ago
comment:16

it would be a good idea to revise this, or perhaps just start from scratch. It's not clear to me whether most/all of this can be achieved with vanilla Sphinx 3 or 4.

mkoeppe commented 4 years ago
comment:17

Related: #30010 - Split sage_setup.docbuild out to a separate distribution (distutils package)

dimpase commented 4 years ago
comment:18

One of sphinx devs, Takeshi KOMIYA, is going to do a comparison with "native" sphinx for the reference manual - https://github.com/sphinx-doc/sphinx/issues/7891

nthiery commented 4 years ago
comment:19

Cool!

mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -1,3 +1,22 @@
+- #30010 Split sage_setup.docbuild out to a separate distribution (distutils package)
+
+- #29868 pip-installable packages `sage-doc-html`, `sage-doc-pdf`
+
+- #30369 Parallel docbuild, cythonization: Use GNU make's POSIX jobserver protocol
+
+- #30475 `sage -docbuild`: error building docs of single file with relative imports
+
+- #6389 expose building documentation for user modules not in the Sage library
+
+
+See also: 
+
+- #20893 Meta-ticket: Improve live documentation in the Jupyter notebook
+
+---
+
+**Overview of Sphinx custom code in Sage's documentation build system**
+
 The goal of this ticket is to write an overview of the various bits and pieces of custom Sphinx code in Sage, with the long term goal to see which pieces could be:
 - thrown away, using features that are now in Sphinx
 - generalized and submitted upstream
jhpalmieri commented 3 years ago

Description changed:

--- 
+++ 
@@ -7,6 +7,8 @@
 - #30475 `sage -docbuild`: error building docs of single file with relative imports

 - #6389 expose building documentation for user modules not in the Sage library
+
+- #30418 Missing PDF icons in the Sage documentation website

 See also:
egourgoulhon commented 3 years ago

Description changed:

--- 
+++ 
@@ -26,7 +26,7 @@

 Here is a list of files with their relative roles:

-**src/doc/common**
+**src/sage/docs**

 - `__init__.py`: empty file
mkoeppe commented 3 years ago
comment:23

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -1,15 +1,14 @@
+Tickets:
+
 - #30010 Split sage_setup.docbuild out to a separate distribution (distutils package)
-
 - #29868 pip-installable packages `sage-doc-html`, `sage-doc-pdf`
-
 - #30369 Parallel docbuild, cythonization: Use GNU make's POSIX jobserver protocol
-
 - #30475 `sage -docbuild`: error building docs of single file with relative imports
-
 - #6389 expose building documentation for user modules not in the Sage library
-
 - #30418 Missing PDF icons in the Sage documentation website
-
+- #28376 Shrink Documentation
+- #31415 GH Actions workflow that deploys built documentation
+- #29868 pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf

 See also:
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -1,6 +1,9 @@
 Tickets:

 - #30010 Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #31353 `sage --docbuild`: Add options to list all documents
+- #31936 docbuild: Japanese characters creeping into other parts of Sage documentation
+- #31948 Reimplement parallel docbuild using `make`
 - #29868 pip-installable packages `sage-doc-html`, `sage-doc-pdf`
 - #30369 Parallel docbuild, cythonization: Use GNU make's POSIX jobserver protocol
 - #30475 `sage -docbuild`: error building docs of single file with relative imports
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -3,6 +3,7 @@
 - #30010 Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #31353 `sage --docbuild`: Add options to list all documents
 - #31936 docbuild: Japanese characters creeping into other parts of Sage documentation
+- #31366 docbuild: switch from `optparse` to `argparse`
 - #31948 Reimplement parallel docbuild using `make`
 - #29868 pip-installable packages `sage-doc-html`, `sage-doc-pdf`
 - #30369 Parallel docbuild, cythonization: Use GNU make's POSIX jobserver protocol
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -35,7 +35,7 @@

 - `conf.py`: standard Sphinx configuration file.

-**src/sage_setup/docbuild**
+**src/sage_docbuild** (was `src/sage_setup/docbuild`)

 - `__init__.py`: top-level docbuilder, contains classes for the various documents (e.g. reference manual).

@@ -47,7 +47,8 @@

    mainly deals with logging and error reporting.

-**src/sage_setup/docbuild/ext**
+**src/sage_docbuild/ext** (was `src/sage_setup/docbuild/ext`)
+

 - `inventory_builder.py`:
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -1,4 +1,4 @@
-Tickets:
+Docbuild tickets:

 - #30010 Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #31353 `sage --docbuild`: Add options to list all documents
@@ -13,6 +13,15 @@
 - #28376 Shrink Documentation
 - #31415 GH Actions workflow that deploys built documentation
 - #29868 pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf
+
+Sphinx tickets:
+
+- #31696 Sphinx/docutils update for Sage 9.4
+- #27578 Use `sphinx.util.inspect.Signature` not `sphinx.ext.autodoc.inspector.formatargspec`
+- #27164 link sphinx docs of Sage components in reference manual
+- #29231 add intersphinx mapping for SciPy
+- #27495 Use `sphinx.ext.extlinks` to add Sphinx roles for links to external package docs
+- #30894 Add typing info to documentation using `sphinx_autodoc_typehints`

 See also:
mkoeppe commented 3 years ago

Description changed:

--- 
+++ 
@@ -21,6 +21,7 @@
 - #27164 link sphinx docs of Sage components in reference manual
 - #29231 add intersphinx mapping for SciPy
 - #27495 Use `sphinx.ext.extlinks` to add Sphinx roles for links to external package docs
+- #30893 Replace sage autodoc extension by built-in one
 - #30894 Add typing info to documentation using `sphinx_autodoc_typehints`

 See also:
mkoeppe commented 2 years ago

Description changed:

--- 
+++ 
@@ -13,6 +13,7 @@
 - #28376 Shrink Documentation
 - #31415 GH Actions workflow that deploys built documentation
 - #29868 pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf
+- https://github.com/sagemath/documentation/issues/24 (Versioned Sage documentation)

 Sphinx tickets: