Build the reference manual incrementally

e14f4152-4982-4ace-8c95-73a0599b109b commented 15 years ago

Building the Sage reference manual can take a significant amount of time. Decreasing this time could speed up Sage development.

The patch is large, but most of it consists of moving files from one location to another, as described below. A summary of the changes:

Changes in doc/en/reference — this is where the size of the patch comes from, although the changes are pretty simple:

rearrange the directory doc/en/reference: for each file like algebras.rst, create a subdirectory algebras and move algebras.rst to algebras/index.rst. Also create a file algebras/conf.py for the build configuration. All of these new conf.py files are identical. Deal with the contents of the directory reference/media similarly, moving the pictures to the appropriate subdirectory.
modify reference/index.rst to point to these new files.
reorganize reference/index.rst so it is arranged, at least somewhat, by topic.
add intersphinx to conf.py — see below. Also add the new subdirectories to the list exclude_trees.
new file conf_sub.py, configuration for the pieces of the documentation (as opposed to the main conf.py, which is for building reference/index.rst). This file is imported by each of the files SUBDIRECTORY/conf.py.

Changes to doc/common/builder.py:

add code to build the reference manual in sections, and also to build the sections in parallel. The reference manual ought to be built twice to resolve references now, so typing "sage -docbuild all html" will build it twice (along with all of the other documents, of course). "sage -docbuild reference html" will just build it once. You can also run "sage -docbuild reference/combinat html", for example, to just build one part of the manual.
the different parts of the manual are separate documents as far as sphinx is concerned, so allow them to reference each other using the "intersphinx" extension. (This is why we need to build it twice: the first pass assembles the intersphinx databases, the second pass uses the databases to create the references correctly.)
to accomodate the changes in #11251, which don't seem to be easily compatible with intersphinx, search through the output files looking for "todo" items, and accumulate them in one master "todo" list.
for pdf format, since it now produces 30 different pdf files, write an html file which links to each of them.

Other changes:

doc/common/conf.py: add the intersphinx extension to the build process.
doc/common/themes/sage/layout.html: fix a bug where clicking the Sage logo in the upper left corner of the docs wouldn't take you to the right place, at least in the local documentation.
doc/common/themes/sageref/: add a new theme for the pieces of the reference manual. This is almost identical to the "sage" theme, except for a little tinkering to the links along the top and bottom lines.
in the main Sage library, change a few pathnames to media files in the reference manual, since those files have been moved.
make the necessary changes to .hgignore and MANIFEST.in to deal with the relocated files.

The html docs for Sage 5.4.rc2, html without MathJax, html with MathJax, and pdf, built after applying the patches here.

Apply:

Before building the docs, you should delete the documentation output directory: rm -rf SAGE_ROOT/devel/sage/doc/output. To test this, you should run sage --docbuild all html and sage --docbuild all pdf. (Note: just running sage --docbuild reference html will probably produce many warnings. If you run it a second time, the warnings should go away.)

Depends on #13064 Depends on #8327 Depends on #13891

Dependencies: 5.7.beta2 + #13064, #8327, #13891

CC: @jhpalmieri @nexttime @nilesjohnson @hivert @mguaypaq @mwhansen

Component: documentation

Keywords: days38

Author: Mitesh Patel, John Palmieri, Florent Hivert

Reviewer: Volker Braun, Florent Hivert

Merged: sage-5.8.beta0

Issue created by migration from https://trac.sagemath.org/ticket/6495

hivert commented 12 years ago

Changed keywords from none to days38

jhpalmieri commented 12 years ago

comment:81

Hi Florent,

I am quite busy with other things right now, so I hope you can get people at Sage Days 38 to take a good look at this. Maybe you can figure out the ctrl-c issue as well; maybe tinkering with what I had might work; if not, the link I provided had some other ideas, too.

Thanks, John

hivert commented 12 years ago

comment:82

Hi John,

Replying to @jhpalmieri:

I am quite busy with other things right now, so I hope you can get people at Sage Days 38 to take a good look at this. Maybe you can figure out the ctrl-c issue as well; maybe tinkering with what I had might work; if not, the link I provided had some other ideas, too.

well, they are very few people knowing the documentation building system here and I used quite a few Sphinx internal stuff. I think a good reviewer would be George Brandle himself but I need to sit for a moment to write him an e-mail.

Cheers,

Florent

mguaypaq commented 12 years ago

comment:83

I haven't had time yet to read the whole ticket description and discussion, but I will note that this patch has allowed me to build the documentation on my old laptop (with only 1GB of RAM) in a couple of hours, whereas trying to build it without the patch causes my laptop to become unresponsive and eventually crash. (Also, Florent had a look at the output and seemed satisfied that everything was fine.) So, a huge +1 from me!

I will try to do as much as I can to review this ticket, but I know almost nothing about the documentation building process.

jhpalmieri commented 12 years ago

comment:85

After a lot of swearing at my computer, I think I have the ctrl-c situation figured out. We've been using Pool.apply_async, and this didn't handle ctrl-c well. Adding a timeout (using the get method) caused it to handle interrupts, but it no longer build in parallel. If we instead use Pool.map_async together with get to provide a timeout, it seems to work. Please take a look at the new patch.

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -32,6 +32,7 @@
 - [attachment: trac_6495-part1-moving-files.patch](https://github.com/sagemath/sage-prod/files/10645464/trac_6495-part1-moving-files.patch.gz) — this moves 'algebras.rst' to 'algebras/index.rst', and similarly for all other files.  It adds `.. include:: ../footer.txt` to the end of each of these files, and it removes any cross-referencing information like `.. _ch:groups:`, since that doesn't work anymore with the new structure.  It also creates identical files 'DIR/conf.py' in each of the new subdirectories of doc/en/reference, except for doc/en/algebras/conf.py.  That file is created in the next patch so that you can focus on reviewing just the second patch.
 - [attachment: trac_6495-part2-everything-else.patch](https://github.com/sagemath/sage-prod/files/10645471/trac_6495-part2-everything-else.patch.gz) — this does everything else; in other words, all of the important content is in this patch.
 - [attachment: trac_6495-part3-the-remaining.patch](https://github.com/sagemath/sage-prod/files/10645448/trac_6495-part3-the-remaining.patch.gz) — this patch takes care of merging the indexes and the bibliography.
+- [attachment: trac_6495-part4-interrupts.patch](https://github.com/sagemath/sage-prod/files/10645450/trac_6495-part4-interrupts.patch.gz) — handle ctrl-c.

 Before building the docs, you should delete the documentation output directory: `rm -rf SAGE_ROOT/devel/sage/doc/output`.

jhpalmieri commented 12 years ago

comment:86

Several quick comments and questions:

First, the combined indices and search are great! At first I thought that searching was broken, but that's my browser, not the code. I'm still looking over the patch, but in practice it seems to work.
This is an old problem: I object to the use of "type" as a global variable, set by __main__. Then I can't use print type(var) anywhere in the rest of the code (as I tried to do when debugging something). I may change type to something else; any suggestions? Perhaps format (which also has a meaning in Python)?
When we're happy about the patches, I'll try to combine them into just one or two chunks. Right now, for example, one of my patches creates the "todolist" directory, and then Florent's patch moves all of the files out of it, so we end up with an empty directory. I don't know if there are other similar things which will need cleaning up.
I get some warnings about missing citations. Should we worry about those now?

jhpalmieri commented 12 years ago

comment:87

I've modified the part 4 patch to do a little cleanup:

I moved the detection of the number of threads to build_options.py, and expanded on the comment about loading build_options.py. This way if someone is reading builder.py and wants to know where NUM_THREADS is defined, at least they'll find a comment pointing them to that file.
I modified the mkdir function, making it a bit more secure: the new approach seems to be preferred, as far as I can tell.

jhpalmieri commented 12 years ago

comment:88

Another update to part 4: fix the main index.html file for the pdf build of the reference manual.

jhpalmieri commented 12 years ago

comment:89

One more update to part 4, to fix a few doctests.

jhpalmieri commented 12 years ago

comment:90

I think this looks very good. I'm just about ready to give Florent's part a positive review. Two questions: can we reinstate the -Q flag for the first pass on the reference manual, to silence all of the warnings? Also, I see this warning message; do you know if it's important?

preparing documents... WARNING: search index couldn't be loaded, but not all documents will be built: the index will be incomplete.

(This occurs if I do sage --docbuild all html, at the end of the second pass through the reference manual.)

I'm attaching one more version of the part 4 patch, just to fix a few typos and grammar issues in the files Florent added.

hivert commented 12 years ago

comment:91

Hi John,

I think this looks very good. I'm just about ready to give Florent's part a positive review.

Wow !!! This is extremely cool. Thanks a lot ! I'm sorry for my current silence. I didn't had the time to look at your part4 code. I'll try to do it shortly.

Two questions: can we reinstate the -Q flag for the first pass on the reference manual, to silence all of the warnings?

No problem. I just wanted to have some idea of the progress and forgot to switch back to a silent mode.

Also, I see this warning message; do you know if it's important?

preparing documents... WARNING: search index couldn't be loaded, but not all documents will be built: the index will be incomplete.

(This occurs if I do sage --docbuild all html, at the end of the second

pass through the reference manual.)

I'm not sure now. I'll though I had silenced this warning. Give me a few day to investigate a little more. It is probably not important as the produced index is correct but I may be missing to merge somme part of it.

I'm attaching one more version of the part 4 patch, just to fix a few typos and grammar issues in the files Florent added.

Thanks for those rereading. I planned to polish more the code and try to get some feedback from Sphinx and didn't find the time. However I now think that we should let the code enter sage as soon as possible because it allows the doc to compile on small machine. At sage days 48, with mguaypaq (see above) we manage to compile the documentation in a seemingly satisfactory way on a 1GB machine. Moreover its a requirement to the feature #12878 which I think is definitely needed at least for huge classes such as graphs...

Thanks a lot,

Florent

jhpalmieri commented 12 years ago

comment:92

See #12991 for a peripherally-related ticket: don't doctest the autogenerated rst files.

jdemeyer commented 12 years ago

comment:93

This needs to be rebased to sage-5.1.beta5:

applying /release/merger/patches/trac_6495-part2-everything-else.patch
patching file doc/en/reference/combinat/index.rst
Hunk #1 FAILED at 3
1 out of 2 hunks FAILED -- saving rejects to file doc/en/reference/combinat/index.rst.rej
patching file doc/en/reference/misc/index.rst
Hunk #1 succeeded at 31 with fuzz 1 (offset 3 lines).
abort: patch failed to apply

jdemeyer commented 12 years ago

Changed dependencies from #12016 to none

jhpalmieri commented 12 years ago

comment:95

Okay, this is now rebased.

jhpalmieri commented 12 years ago

comment:96

I've added a link to the built documents for Sage 5.1.beta5, in case anyone wants to look at it without applying the patches here. The reference manual is organized differently and as a result looks different. The other documents should be unchanged.

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -27,6 +27,10 @@

 ---

+[The html docs for Sage 5.1.beta5](http://sage.math.washington.edu/home/palmieri/misc/6495/html/en/), built after applying the patches here.
+
+---
+
 Apply:

 - [attachment: trac_6495-part1-moving-files.patch](https://github.com/sagemath/sage-prod/files/10645464/trac_6495-part1-moving-files.patch.gz) — this moves 'algebras.rst' to 'algebras/index.rst', and similarly for all other files.  It adds `.. include:: ../footer.txt` to the end of each of these files, and it removes any cross-referencing information like `.. _ch:groups:`, since that doesn't work anymore with the new structure.  It also creates identical files 'DIR/conf.py' in each of the new subdirectories of doc/en/reference, except for doc/en/algebras/conf.py.  That file is created in the next patch so that you can focus on reviewing just the second patch.
@@ -35,4 +39,3 @@
 - [attachment: trac_6495-part4-interrupts.patch](https://github.com/sagemath/sage-prod/files/10645450/trac_6495-part4-interrupts.patch.gz) — handle ctrl-c.

 Before building the docs, you should delete the documentation output directory: `rm -rf SAGE_ROOT/devel/sage/doc/output`.
-

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -27,7 +27,9 @@

 ---

-[The html docs for Sage 5.1.beta5](http://sage.math.washington.edu/home/palmieri/misc/6495/html/en/), built after applying the patches here.
+The html docs for Sage 5.1.beta5, built
+[without jsMath](http://sage.math.washington.edu/home/palmieri/misc/6495/html/en/) and
+[with jsMath](http://sage.math.washington.edu/home/palmieri/misc/6495-jsmath/html/en/), built after applying the patches here.

 ---

jhpalmieri commented 12 years ago

Dependencies: #9774

jhpalmieri commented 12 years ago

comment:98

This now depends on #9774. If you want to try it without the patches at #9774 (thus using jsMath instead of MathJax), apply attachment: trac_6495-part2-everything-else.patch instead of attachment: trac_6495-part2-everything-else-9774.patch.

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -36,7 +36,7 @@
 Apply:

 - [attachment: trac_6495-part1-moving-files.patch](https://github.com/sagemath/sage-prod/files/10645464/trac_6495-part1-moving-files.patch.gz) — this moves 'algebras.rst' to 'algebras/index.rst', and similarly for all other files.  It adds `.. include:: ../footer.txt` to the end of each of these files, and it removes any cross-referencing information like `.. _ch:groups:`, since that doesn't work anymore with the new structure.  It also creates identical files 'DIR/conf.py' in each of the new subdirectories of doc/en/reference, except for doc/en/algebras/conf.py.  That file is created in the next patch so that you can focus on reviewing just the second patch.
-- [attachment: trac_6495-part2-everything-else.patch](https://github.com/sagemath/sage-prod/files/10645471/trac_6495-part2-everything-else.patch.gz) — this does everything else; in other words, all of the important content is in this patch.
+- [attachment: trac_6495-part2-everything-else-9774.patch](https://github.com/sagemath/sage-prod/files/10645449/trac_6495-part2-everything-else-9774.patch.gz) — this does everything else; in other words, all of the important content is in this patch.  (If you haven't applied the patches at #9774, apply [attachment: trac_6495-part2-everything-else.patch](https://github.com/sagemath/sage-prod/files/10645471/trac_6495-part2-everything-else.patch.gz) instead.)
 - [attachment: trac_6495-part3-the-remaining.patch](https://github.com/sagemath/sage-prod/files/10645448/trac_6495-part3-the-remaining.patch.gz) — this patch takes care of merging the indexes and the bibliography.
 - [attachment: trac_6495-part4-interrupts.patch](https://github.com/sagemath/sage-prod/files/10645450/trac_6495-part4-interrupts.patch.gz) — handle ctrl-c.

jhpalmieri commented 12 years ago

comment:99

A slight issue with the reference manual is that it takes up a little more room, and most of the difference is in devel/sage/doc/output/doctrees. There is some redundancy in the environment.pickle files. I wonder if there is any way to avoid this.

In more detail: using MathJax instead of jsMath saves room in the directory doc/output/html: with the old version, jsMath and non-parallel, that directory takes about 325M, while the new version, MathJax and parallel, takes about 200M, saving 125M. Unfortunately, the doctree directory has increased by about 150M, so the whole thing has gone up by 25M. Not a big difference, but if there were a way to improve it, it would be nice. Maybe on another ticket...

jdemeyer commented 12 years ago

Changed dependencies from #9774 to #13126, #11078, #11874, #12229, #11503, #12327, #11080, #9774, #8473, #11913, #12299, #13121

jdemeyer commented 12 years ago

comment:100

Clarified recursive dependencies.

jdemeyer commented 12 years ago

Work Issues: too many WARNINGs

jdemeyer commented 12 years ago

comment:101

I get tons of warnings (1043 to be precise) when building the manuals. I don't like this, because it's hard to distinguish warnings due to this ticket with true warnings about bad formatting. Is there a way not to produce any warnings during the first run (I assume the warnings should only appear in the first run)?

jhpalmieri commented 12 years ago

comment:102

Rebased because of #12299. I'll try to work on the warnings. I think it might be best to only suppress the warnings when you run sage --docbuild all html, because that is when both passes of the reference manual are actually run, and the first pass is designed to be fast: it just produces the inventory files, not html output. If you run sage --docbuild reference html, then it will just do one pass, html format.

Edit: actually, the warnings are easy to get rid of. Florent intentionally left them there while working on the ticket: see these lines in builder.py:

        global ALLSPHINXOPTS
        # ALLSPHINXOPTS += ' -Q -D multidoc_first_pass=1'
        ALLSPHINXOPTS += ' -D multidoc_first_pass=1'

If we uncomment the second line and remove the third, the warnings will go away. I'll incorporate this change into the 4th patch.

jhpalmieri commented 12 years ago

Changed work issues from too many WARNINGs to none

jhpalmieri commented 12 years ago

comment:105

I see one warning now:

WARNING: search index couldn't be loaded, but not all documents will be built: the index will be incomplete.

I don't know what this means. The search index looks pretty good to me. Florent might have some idea, when he has a chance to look at this.

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -33,6 +33,10 @@

 ---

+[Here](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495/) is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.1.beta6, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495.tar) is the corresponding tar file. Note that the sagenb spkg from #13121 is a little bit broken, so the functionality from #8473 is not actually functional. I think you also need to install the new [JMOL spkg](http://www.uwosh.edu/faculty_staff/gutow/jmol-12.3.27.p0.spkg) by hand: after building Sage, run `./sage -i http://www.uwosh.edu/faculty_staff/gutow/jmol-12.3.27.p0.spkg`.
+
+---
+
 Apply:

 - [attachment: trac_6495-part1-moving-files.patch](https://github.com/sagemath/sage-prod/files/10645464/trac_6495-part1-moving-files.patch.gz) — this moves 'algebras.rst' to 'algebras/index.rst', and similarly for all other files.  It adds `.. include:: ../footer.txt` to the end of each of these files, and it removes any cross-referencing information like `.. _ch:groups:`, since that doesn't work anymore with the new structure.  It also creates identical files 'DIR/conf.py' in each of the new subdirectories of doc/en/reference, except for doc/en/algebras/conf.py.  That file is created in the next patch so that you can focus on reviewing just the second patch.

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -33,7 +33,7 @@

 ---

-[Here](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495/) is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.1.beta6, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495.tar) is the corresponding tar file. Note that the sagenb spkg from #13121 is a little bit broken, so the functionality from #8473 is not actually functional. I think you also need to install the new [JMOL spkg](http://www.uwosh.edu/faculty_staff/gutow/jmol-12.3.27.p0.spkg) by hand: after building Sage, run `./sage -i http://www.uwosh.edu/faculty_staff/gutow/jmol-12.3.27.p0.spkg`.
+[Here](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495/) is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.1.beta6, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495.tar) is the corresponding tar file. Note that the sagenb spkg from #13121 is a little bit broken, so the functionality from #8473 is not actually functional.

 ---

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -33,7 +33,7 @@

 ---

-[Here](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495/) is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.1.beta6, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495.tar) is the corresponding tar file. Note that the sagenb spkg from #13121 is a little bit broken, so the functionality from #8473 is not actually functional.
+[Here](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495/) is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.1.beta6, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495.tar) is the corresponding tar file. Note that the sagenb spkg from #13121 is a little bit broken, so the functionality from #8473 is not actually functional. The spkg is also broken in another way, so before building Sage, cd to SAGE_ROOT and do `mkdir devel`.

 ---

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -33,7 +33,7 @@

 ---

-[Here](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495/) is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.1.beta6, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.b6-6495.tar) is the corresponding tar file. Note that the sagenb spkg from #13121 is a little bit broken, so the functionality from #8473 is not actually functional. The spkg is also broken in another way, so before building Sage, cd to SAGE_ROOT and do `mkdir devel`.
+[Here](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.rc0-6495/) is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.1.rc0, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.rc0-6495.tar) is the corresponding tar file. Note that the sagenb spkg from #13121 is a little bit broken, so the functionality from #8473 is not actually functional.

 ---

jhpalmieri commented 12 years ago

comment:110

In addition to the warning mentioned in comment #105, I have a question about .buildinfo files. It seems that on the first pass through the reference manual, these files contain lines like

config: 6a23e6beb735e39dc46994bfb813cf55
tags: fbb0d17656682115ca4d033fb2f83ba1

On the second pass, these files get overwritten, and the new files have

config:
tags:

Then running sage --docbuild reference all produces warnings like

WARNING: unsupported build info format in '.../devel/sage/doc/output/html/en/reference/libs/.buildinfo', building all

and then everything is rebuilt again. I propose this change, which seems to fix this issue:

diff --git a/doc/common/builder.py b/doc/common/builder.py
--- a/doc/common/builder.py
+++ b/doc/common/builder.py
@@ -300,7 +300,7 @@ class AllBuilder(object):
         logger.warning("Building reference manual, second pass.\n")
         ALLSPHINXOPTS = ALLSPHINXOPTS.replace(
             'multidoc_first_pass=1', 'multidoc_first_pass=0')
-        ALLSPHINXOPTS = ALLSPHINXOPTS.replace('-Q', '-q') + ' -a '
+        ALLSPHINXOPTS = ALLSPHINXOPTS.replace('-Q', '-q') + ' '
         for document in refs:
             getattr(get_builder(document), name)(*args, **kwds)

jhpalmieri commented 12 years ago

apply second (if you've applied the patches at #9774)

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -39,6 +39,10 @@

 Apply:

+- [attachment: trac_6495-all-in-one.patch](https://github.com/sagemath/sage-prod/files/10645461/trac_6495-all-in-one.patch.gz).
+
+For ease of review, you can instead look at the following four patches:
+
 - [attachment: trac_6495-part1-moving-files.patch](https://github.com/sagemath/sage-prod/files/10645464/trac_6495-part1-moving-files.patch.gz) — this moves 'algebras.rst' to 'algebras/index.rst', and similarly for all other files.  It adds `.. include:: ../footer.txt` to the end of each of these files, and it removes any cross-referencing information like `.. _ch:groups:`, since that doesn't work anymore with the new structure.  It also creates identical files 'DIR/conf.py' in each of the new subdirectories of doc/en/reference, except for doc/en/algebras/conf.py.  That file is created in the next patch so that you can focus on reviewing just the second patch.
 - [attachment: trac_6495-part2-everything-else-9774.patch](https://github.com/sagemath/sage-prod/files/10645449/trac_6495-part2-everything-else-9774.patch.gz) — this does everything else; in other words, all of the important content is in this patch.  (If you haven't applied the patches at #9774, apply [attachment: trac_6495-part2-everything-else.patch](https://github.com/sagemath/sage-prod/files/10645471/trac_6495-part2-everything-else.patch.gz) instead.)
 - [attachment: trac_6495-part3-the-remaining.patch](https://github.com/sagemath/sage-prod/files/10645448/trac_6495-part3-the-remaining.patch.gz) — this patch takes care of merging the indexes and the bibliography.

jhpalmieri commented 12 years ago

comment:111

Attachment: trac_6495-part2-everything-else-9774.patch.gz

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -33,7 +33,7 @@

 ---

-[Here](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.rc0-6495/) is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.1.rc0, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.1.rc0-6495.tar) is the corresponding tar file. Note that the sagenb spkg from #13121 is a little bit broken, so the functionality from #8473 is not actually functional.
+[hhttp://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.2.beta0-6495/ Here] is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.2.beta0, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.2.beta0-6495.tar) is the corresponding tar file. 

 ---

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -45,7 +45,7 @@

 - [attachment: trac_6495-part1-moving-files.patch](https://github.com/sagemath/sage-prod/files/10645464/trac_6495-part1-moving-files.patch.gz) — this moves 'algebras.rst' to 'algebras/index.rst', and similarly for all other files.  It adds `.. include:: ../footer.txt` to the end of each of these files, and it removes any cross-referencing information like `.. _ch:groups:`, since that doesn't work anymore with the new structure.  It also creates identical files 'DIR/conf.py' in each of the new subdirectories of doc/en/reference, except for doc/en/algebras/conf.py.  That file is created in the next patch so that you can focus on reviewing just the second patch.
 - [attachment: trac_6495-part2-everything-else-9774.patch](https://github.com/sagemath/sage-prod/files/10645449/trac_6495-part2-everything-else-9774.patch.gz) — this does everything else; in other words, all of the important content is in this patch.  (If you haven't applied the patches at #9774, apply [attachment: trac_6495-part2-everything-else.patch](https://github.com/sagemath/sage-prod/files/10645471/trac_6495-part2-everything-else.patch.gz) instead.)
-- [attachment: trac_6495-part3-the-remaining.patch](https://github.com/sagemath/sage-prod/files/10645448/trac_6495-part3-the-remaining.patch.gz) — this patch takes care of merging the indexes and the bibliography.
+- [attachment: trac_6495-part3-the-remaining-vs-5.4.patch](https://github.com/sagemath/sage-prod/files/10645451/trac_6495-part3-the-remaining-vs-5.4.patch.gz) — this patch takes care of merging the indexes and the bibliography.
 - [attachment: trac_6495-part4-interrupts.patch](https://github.com/sagemath/sage-prod/files/10645450/trac_6495-part4-interrupts.patch.gz) — handle ctrl-c.

 Before building the docs, you should delete the documentation output directory: `rm -rf SAGE_ROOT/devel/sage/doc/output`.

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -44,7 +44,7 @@
 For ease of review, you can instead look at the following four patches:

 - [attachment: trac_6495-part1-moving-files.patch](https://github.com/sagemath/sage-prod/files/10645464/trac_6495-part1-moving-files.patch.gz) — this moves 'algebras.rst' to 'algebras/index.rst', and similarly for all other files.  It adds `.. include:: ../footer.txt` to the end of each of these files, and it removes any cross-referencing information like `.. _ch:groups:`, since that doesn't work anymore with the new structure.  It also creates identical files 'DIR/conf.py' in each of the new subdirectories of doc/en/reference, except for doc/en/algebras/conf.py.  That file is created in the next patch so that you can focus on reviewing just the second patch.
-- [attachment: trac_6495-part2-everything-else-9774.patch](https://github.com/sagemath/sage-prod/files/10645449/trac_6495-part2-everything-else-9774.patch.gz) — this does everything else; in other words, all of the important content is in this patch.  (If you haven't applied the patches at #9774, apply [attachment: trac_6495-part2-everything-else.patch](https://github.com/sagemath/sage-prod/files/10645471/trac_6495-part2-everything-else.patch.gz) instead.)
+- [attachment: trac_6495-part2-everything-else.patch](https://github.com/sagemath/sage-prod/files/10645471/trac_6495-part2-everything-else.patch.gz) — this does everything else; in other words, all of the important content is in this patch.
 - [attachment: trac_6495-part3-the-remaining-vs-5.4.patch](https://github.com/sagemath/sage-prod/files/10645451/trac_6495-part3-the-remaining-vs-5.4.patch.gz) — this patch takes care of merging the indexes and the bibliography.
 - [attachment: trac_6495-part4-interrupts.patch](https://github.com/sagemath/sage-prod/files/10645450/trac_6495-part4-interrupts.patch.gz) — handle ctrl-c.

jhpalmieri commented 12 years ago

comment:115

Rebased to Sage 5.4.beta0.

jhpalmieri commented 12 years ago

Description changed:

--- 
+++ 
@@ -27,13 +27,9 @@

 ---

-The html docs for Sage 5.1.beta5, built
-[without jsMath](http://sage.math.washington.edu/home/palmieri/misc/6495/html/en/) and
-[with jsMath](http://sage.math.washington.edu/home/palmieri/misc/6495-jsmath/html/en/), built after applying the patches here.
-
----
-
-[hhttp://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.2.beta0-6495/ Here] is a distribution including this ticket and all of its prerequisites, built on top of Sage 5.2.beta0, and [this](http://sage.math.washington.edu/home/palmieri/misc/6495/sage-5.2.beta0-6495.tar) is the corresponding tar file. 
+The html docs for Sage 5.4.beta0, built
+[without MathJax](http://sage.math.washington.edu/home/palmieri/misc/6495/html/en/) and
+[with MathJax](http://sage.math.washington.edu/home/palmieri/misc/6495-jsmath/html/en/), built after applying the patches here.

 ---

jhpalmieri commented 12 years ago

Description changed:

jhpalmieri commented 12 years ago

comment:118

Florent: any chance you can work on this? I think that the only thing needed reviewing is the part 4 patch. (There is also the warning at comment 105, but I think that the actual output is good, so suppressing the warning would be nice but not absolutely necessary. You might also look at comment 110, but I can't reproduce that issue right now.)

hivert commented 12 years ago

comment:119

Hi John,

Replying to @jhpalmieri:

Florent: any chance you can work on this? I think that the only thing needed reviewing is the part 4 patch. (There is also the warning at comment 105, but I think that the actual output is good, so suppressing the warning would be nice but not absolutely necessary. You might also look at comment 110, but I can't reproduce that issue right now.)

Yes ! And thanks for your work and your patience. This is very high on my priority list, but it has to remain after the list "things that must be done right now" (such as teaching or meeting the dead-line to get money for Sage-Combinat) or even "thing that should have been done yesterday". Unfortunately, until the end of September, I will still be moving a flat, so I definitely won't be able to work during the Week-End. I definitely want to do this done ASAP. So I didn't give-up, but the summer didn't went as expected and I had to stop working on sage for a few month. Please ping me in the first week of October if I didn't manage to do it .

Cheers,

Florent

jhpalmieri commented 12 years ago

comment:120

Rebased to Sage 5.4.beta1.

jhpalmieri commented 12 years ago

Changed dependencies from #13126, #11078, #11874, #12229, #11503, #12327, #11080, #9774, #8473, #11913, #12299, #13121 to #13143

jhpalmieri commented 12 years ago

comment:121

Made #13143 a dependency and rebased to that.

hivert commented 12 years ago

comment:122

Hi John,

Replying to @jhpalmieri:

Made #13143 a dependency and rebased to that.

Consider me as pinged. And don't hesitate to email me more aggressively if nothing come up before the end of the week.

Florent

sagemath / sage

Build the reference manual incrementally #6495