texinfo documentation for sage

mantepse commented 8 years ago

texinfo is the native documentation format to emacs. sphinx can output texinfo. we should use that.

CC: stakemorii@gmail.com @hivert

Component: documentation

Keywords: Sphinx

Author: Martin Rubey

Branch/Commit: u/mantepse/texinfo_documentation_for_sage @ b3dac9d

Issue created by migration from https://trac.sagemath.org/ticket/21734

mantepse commented 8 years ago

Branch: u/mantepse/texinfo_documentation_for_sage

mantepse commented 8 years ago

Description changed:

--- 
+++ 
@@ -1 +1 @@
-
+texinfo is the native documentation format to emacs.  sphinxs can output texinfo.  we should use that.

mantepse commented 8 years ago

comment:2

applying this patch and doing ./sage -docbuild reference texinfo leaves a .texi file and a Makefile in each directory below SAGEROOT/local/share/doc/sage/texinfo/en/reference. Issuing make in each directory then creates a beautiful .info file, which can be visited in emacs using C-u C-h i filename.

Currently I do not know how to make sage make the .info files itself, and I do not know how sage-shell-mode could display them.

New commits:

556e8d3 create texi files

mantepse commented 8 years ago

Commit: 556e8d3

mantepse commented 8 years ago

Author: Martin Rubey

mantepse commented 8 years ago

Description changed:

--- 
+++ 
@@ -1 +1 @@
-texinfo is the native documentation format to emacs.  sphinxs can output texinfo.  we should use that.
+texinfo is the native documentation format to emacs.  sphinx can output texinfo.  we should use that.

mantepse commented 8 years ago

comment:4

I am having some problems with the texinfo-hyperlinks. For example, in

 sage: ASM = AlternatingSignMatrices(3)

the (3) is interpreted as a link to a footnote. The corresponding bit of the texinfo source looks OK, though:

@geindex from_contre_tableau() (sage.combinat.alternating_sign_matrix.AlternatingSignMatrices method)
@anchor{sage/combinat/alternating_sign_matrix sage combinat alternating_sign_matrix AlternatingSignMatrices from_contre_tableau}@anchor{227}
@deffn {Method} from_contre_tableau (comps)

Return an alternating sign matrix from a contre-tableau.

EXAMPLES:

@example
sage: ASM = AlternatingSignMatrices(3)
sage: ASM.from_contre_tableau([[1, 2, 3], [1, 2], [1]])
[0 0 1]
[0 1 0]
[1 0 0]
sage: ASM.from_contre_tableau([[1, 2, 3], [2, 3], [3]])
[1 0 0]
[0 1 0]
[0 0 1]
@end example
@end deffn

mantepse commented 8 years ago

comment:5

Replying to @mantepse:

I am having some problems with the texinfo-hyperlinks. For example, in
 sage: ASM = AlternatingSignMatrices(3)
the (3) is interpreted as a link to a footnote. The corresponding bit of the texinfo source looks OK, though:

in the standalone info program, this problem does not appear. There are some hints in the Sphinx FAQ http://www.sphinx-doc.org/en/stable/faq.html#texinfo-info but I do not see why they would apply.

dc671f89-e3b6-4e57-8c31-31a328d54e38 commented 8 years ago

comment:6

Replying to @mantepse:

Thank you for the patch. I attended a conference this week and will try the patch when I go back home.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

66f2f43 also build info files

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Changed commit from 556e8d3 to 66f2f43

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Changed commit from 66f2f43 to 7012f33

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

7012f33 give the info file the same name as the directory

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

4294647 simplify (images are still not copied)

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Changed commit from 7012f33 to 4294647

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

db003fa more sensible values in texinfo_documents

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Changed commit from 4294647 to db003fa

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

`6774f4f`	`remove line breaks in rubrics titles`
`491ad5e`	`repair removal of newlines`

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Changed commit from db003fa to 491ad5e

mantepse commented 8 years ago

comment:12

Florent, if you could help with the problem reported at https://groups.google.com/forum/#!topic/sphinx-users/-8cHf1ZD5S4 that would be great. This seems very much related to #9128!

mantepse commented 8 years ago

Changed keywords from none to Sphinx

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

c73c232 not sure, but I think this needs to set texinfo_documents, too

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 8 years ago

Changed commit from 491ad5e to c73c232

mantepse commented 8 years ago

comment:14

Dear Sho,

unfortunately, I still have no clue how to enable the crossreferences between the individual manuals. I asked on the sphinx user group https://groups.google.com/forum/#!topic/sphinx-users/-8cHf1ZD5S4 and also on the sphinx github page https://github.com/sphinx-doc/sphinx/issues/3089, but have not got any response so far.

meanwhile, I discovered SAGEROOT/src/doc/common/conf.py, which contains the call app.connect('missing-reference', find_sage_dangling_links).

I suspect that this (I mean find_sage_dangling_links) is where I should put the proper handling for texinfo crossreferences, but I'm not sure yet...

dc671f89-e3b6-4e57-8c31-31a328d54e38 commented 8 years ago

comment:15

Replying to @mantepse:

It is already convenient to view Sage document via info. Thank you very much. Concerning with sphinx, I cannot help you, sorry.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 6 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

b3dac9d Merge branch 'develop' of git://trac.sagemath.org/sage into t/21734/texinfo_documentation_for_sage

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 6 years ago

Changed commit from c73c232 to b3dac9d

jhpalmieri commented 6 years ago

comment:17

The reference manual doesn't build for me (html, not texinfo) without this change:

diff --git a/src/doc/en/reference/conf.py b/src/doc/en/reference/conf.py
index 5c7bbe95ec..22961d90ae 100644
--- a/src/doc/en/reference/conf.py
+++ b/src/doc/en/reference/conf.py
@@ -67,7 +67,7 @@ latex_elements['preamble'] += r'''
 # @end direntry

 texinfo_documents = [
-('index', name, title, u'The Sage Development Team', name, title, 'SageMath')
+('index', name, project, u'The Sage Development Team', name, project, 'SageMath')
 ]

 #Ignore all .rst in the _sage subdirectory

or something similar: title is not defined in reference/conf.py.

jhpalmieri commented 6 years ago

comment:18

You could also consider adding a Makefile target:

diff --git a/build/make/deps b/build/make/deps
index 371a56a767..65ad3fdc07 100644
--- a/build/make/deps
+++ b/build/make/deps
@@ -247,6 +247,8 @@ doc-src-clean:
 doc-output-clean:
        rm -rf "$(SAGE_SHARE)/doc/sage"

+doc-texinfo: $(DOC_DEPENDENCIES)
+       $(AM_V_at)cd ../.. && sage-logger -p './sage --docbuild all texinfo $(SAGE_DOCBUILD_OPTS)' logs/doctexinfo.log

 ###############################################################################
 # Cleaning up

(After doing this, make doc-texinfo fails for me when building the developer's guide, by the way.) Then someone who is better at make than I am could possibly add a target to run make in all of the subdirectories of local/share/doc/sage/texinfo. It could be a bit of a mess, but we could instead set $infodir to point to local/share/info and then run make install-info instead of make. The problem is that texinfo/de/tutorial/sagetutorial.texi and texinfo/en/tutorial/sagetutorial.texi will produce .info files of the same name. Does info handle subdirectories well? Should we have local/share/info/de/ or local/share/info/sage/de/, etc.? The Sage reference manual will also consist of lots of separate .info files; how should those be handled?

mantepse commented 6 years ago

comment:19

Thank you for looking into this!

I'm afraid, however, that there is a shostopper: I couldn't get the links between the individual texinfo files to work. I filed https://github.com/sphinx-doc/sphinx/issues/3089 but it was closed. The advice was: "Please integrate your project into single one."

It may well be that I misunderstood the whole thing - but as far as I could tell at the time, the sage documentation consists of many subprojects, one for each folder below src/sage. If that's not the case, that would be great and I'd be happy to learn more about it!

aikrahguzar commented 3 months ago

@mantepse a texinfo manual would be great and I would like to help but I have little familiarity with texinfo, sphinx and sage's docbuilder. I would like to try a working manual but my chances of succeeding are dim.

I think the problems are:

Links between different manuals produced by sage don't work.
Links to sage classes with a manual don't work. I think this might be an info documentation? E.g. even in Emacs I don't now how to refer to the location of a function's description rather than the node containing that description.
What to do about math in Sage manuals? It seems to be included verbatim in the manual. Looking at the texinfo documentation, it seems like math is handled when producing a pdf but not when using Emacs or standalone info viewer.

I try your work and extend it but it will probably take a lot of time. Thanks for the foundation.

aikrahguzar commented 3 months ago

@mantepse maybe the suggestion to integrate the project into a single one is worth considering. Not as a change to the sage project but for the more restricted purpose of building an info manual.

With this diff

@@ -75,6 +75,8 @@ multidocs_subdoc_list = sorted([x for x in os.listdir(ref_src)

 # List of directories, relative to source directory, that shouldn't be
 # searched for source files.
-exclude_patterns += multidocs_subdoc_list + [
-    'sage', 'options'
-    ]
+# exclude_patterns += multidocs_subdoc_list + [
+#     'sage', 'options'
+#     ]
+
+exclude_patterns = ['_sage']

to src/doc/en/reference/conf.py, I am able to build a usable manual for the reference by running

./sage -sh -c "sphinx-build -M info ./src/doc/en/reference ../info"

from the directory with sage repo. This command actually fails after producing the texi due to errors like

@menu reference to nonexistent node `b7'

and I have to run from the directory containing the texi

makeinfo --no-split --force -o 'referencemanual.info' 'referencemanual.texi'

to produce the info file.

A lot of cross references don't work but I am not sure why, since they are defined as @anchor in the texi file and in the info file I can see them in lines like,

Ref: anchor_id^?a_large_number

Do you have any idea? Might be due to the fact that there are multiple anchors for the same location in the texi? I messes up the navigation pretty badly but for me at least the manual is usable with Info-goto-node.

aikrahguzar commented 3 months ago

The cross references actually work. Here is what I did: I used open the info file then M-x Info-on-current-buffer RET. At this stage cross references don't work, however they start working after reverting the buffer. I don't know why.

Another thing, in addition to conf.py above, src/doc/en/reference/index.rst also needs to be modified,

@@ -183,3 +183,9 @@ Indices and Tables

 .. math::
    :nowrap:
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   */index

mantepse commented 3 months ago

I find it very cool that you are obviously making progress! Do you think you could prepare a pull request at some point so that it is easy to follow your work?

I use emacs and sage-shell-mode (and nothing else), so I'd be thrilled to see this working!

aikrahguzar commented 3 months ago

I find it very cool that you are obviously making progress! Do you think you could prepare a pull request at some point so that it is easy to follow your work?

I don't if there is a path for this hack to be upstreamed and I still don't understand how the sage build process works. Maybe conditionally excluding directories depending on the format will be acceptable for conf.py but I think that toctree in the index.rst is going to be problematic. However I don't understand sphinx enough to say anything with confidence.

I use emacs and sage-shell-mode (and nothing else), so I'd be thrilled to see this working!

Me too, though I feel I can get by with the hack I have in place now and build the documentation myself. Anyhow here is the gzipped manual I was able to obtain. You can download it and M-x Info-on-current-buffer to see how it looks. It has (I think) everything in the English section. Some things are not ideal (like too many Indices and Tables sections in (sage.info) Indices and Tables) but for me it is an improvement over the html one.

sage.info.gz

mantepse commented 3 months ago

I find it very cool that you are obviously making progress! Do you think you could prepare a pull request at some point so that it is easy to follow your work?

I don't if there is a path for this hack to be upstreamed and I still don't understand how the sage build process works. Maybe conditionally excluding directories depending on the format will be acceptable for conf.py but I think that toctree in the index.rst is going to be problematic. However I don't understand sphinx enough to say anything with confidence.

Even if it's not complete, it would be cool if you could create a pull request that contains the necessary modifications (if there are any), and a step by step list how to recreate sage.info.gz. It is quite possible that other people can continue from there.

aikrahguzar commented 3 months ago

@mantepse I created a branch https://github.com/aikrahguzar/sage/tree/info-manual, I don't want to create a draft pr since it is based off a branch which is not merged yet but which I need to build on my computer.

To build the info manual,

# Generate src/doc/en/reference/repl/options.txt
cd src/doc/
make doc-src
cd -
# Generate inventory
./sage --docbuild reference inventory
# Generate texi (replace target by the directory where build artifacts should go.)
./sage -sh -c "sphinx-build -b texinfo ./src/doc/en target"
# Generate info
makeinfo --no-split --force -o target/sage.info target/sage.texi

The --force argument can be removed by adding texinfo_domain_indices = False to the src/doc/en/conf.py but that removes the python module index from the manual which I think is worse.

I am mostly satisfied with the state here. This setup allows for generating an info manual without messing with the regular build. However if someone wants to continue, here are some thing to figure out:

Anyway to remove the --force? I think the problem is with the generation of texi which includes lines like,
```
* sage.algebras.affine_nil_temperley_lieb: 0. 
```
and the problem that 0. on the right. Other nodes don't any such numbers.
Is there a way to get rid of nodes like (sage) Indices and Tables<2>? They are empty or refer to html files and aren't needed.
The top node can be better organized if done manually.

One problem I would like to solve is the math rendering. But that probably requires changes in texinfo so unlikely in the short term.

mantepse commented 3 months ago

Hi @aikrahguzar, thanks for sharing! I created a local branch that only contains your two commits with the modifications for texinfo. If you don't mind, I would turn it into a pull request.

Did you have a look at my (ancient) modifications, i.e., u/mantepse/texinfo_documentation_for_sage? Have they become unnecessary? I do not remember what I did and what it was good for, unfortunately.

Browsing through the info file your procedure produces, it seems that the structure of the documentation could be improved quite a bit. There are also a few issues with the typesetting - the current doc in sage-emacs-mode does a good job there (concerning colors and stripping unwanted information).

Also, I really would like to be able to use

sage: LazySymmetricFunctions?

to jump to the correct node in the info manual.

aikrahguzar commented 3 months ago

Hi @aikrahguzar, thanks for sharing! I created a local branch that only contains your two commits with the modifications for texinfo. If you don't mind, I would turn it into a pull request.

Please go ahead.

Did you have a look at my (ancient) modifications, i.e., u/mantepse/texinfo_documentation_for_sage? Have they become unnecessary? I do not remember what I did and what it was good for, unfortunately.

My guess is they are not needed. I don't understand them well enough but I think they were trying to use the sage docbuild system for building info manuals. This bypasses that and uses sphinx-build directly. There were others which addressed a couple of issues mentioned in https://github.com/sphinx-doc/sphinx/issues/3089 but those have been addressed in sphinx.

Browsing through the info file your procedure produces, it seems that the structure of the documentation could be improved quite a bit.

Agreed, I think some of the issues could be addressed by a handwritten src/doc/en/index.rst. That might be good enough to get rid of the Indices and Tables entries which I think are the biggest issues.

There are also a few issues with the typesetting - the current doc in sage-emacs-mode does a good job there (concerning colors and stripping unwanted information).

The only typesetting problem I see is how the latex math code is copied verbatim. I sent en email to the info help mailing list about it.

Coloring Class, Method, sage prompts etc should be possible with a custom jit-lock-function. I might try my hand at that and will let you know if I do.

But even the Emacs manual doesn't do much coloring and I find the look alright there.

Also, I really would like to be able to use
sage: LazySymmetricFunctions?
to jump to the correct node in the info manual.

I think this is very doable. Use a custom comint-input-sender to capture the input ending with ?. Compute the fully qualified name of the identifier and then Info-goto-node("(sage)NODE-NAME") where NODE-NAME is computed from the fully qualified name.

aikrahguzar commented 3 months ago

There are also a few issues with the typesetting - the current doc in sage-emacs-mode does a good job there (concerning colors and stripping unwanted information).

I took a stab at the fontification here https://codeberg.org/rahguzar/sage-mode/src/commit/361e11ded046d5ffbb1e4eb89b7590c44dcf5e49/sage.el#L528-L618

It can be trivially adapted to sage-shell-mode by replacing one call to sage-mode with the one provided by sage-shell-mode and changing the variable names etc to fit the convention there.

I also asked on the mailing list for gnu-info help and we might get the math parts of the manual as typeset images at some point https://lists.gnu.org/archive/html/help-texinfo/2024-08/msg00003.html

sagemath / sage

texinfo documentation for sage #21734