Use sage_getdoc in sage interpreter when doing "?"

[simon-king-jena]

The purpose of this ticket is to fix two bugs.

First problem

Theere is a bug in sage.misc.sagedoc.format, that ironically makes that format? does not show the documentation of format.

sage: from sage.misc.sagedoc import format
sage: format.__doc__
'\n    Format Sage documentation ``s`` for viewing with IPython.\n\n    This calls ``detex`` on ``s`` to convert LaTeX commands to plain\n    text, and if ``s`` contains a string of the form "<<<obj>>>",\n    then it replaces it with the docstring for "obj".\n\n    INPUT:\n\n    - ``s`` - string\n    - ``embedded`` - boolean (optional, default False)\n\n    OUTPUT: string\n\n    Set ``embedded`` equal to True if formatting for use in the\n    notebook; this just gets passed as an argument to ``detex``.\n\n    EXAMPLES::\n\n        sage: from sage.misc.sagedoc import format\n        sage: identity_matrix(2).rook_vector.__doc__[115:184]\n        \'Let `A` be a general `m` by `n`\\n        (0,1)-matrix with `m \\\\le n`. \'\n        sage: format(identity_matrix(2).rook_vector.__doc__[115:184])\n        \'Let A be a general m by n\\n   (0,1)-matrix with m <= n.\\n\'\n\n    If the first line of the string is \'nodetex\', remove \'nodetex\' but\n    don\'t modify any TeX commands::\n    \n        sage: format("nodetex\\n`x \\\\geq y`")\n        \'\\n`x \\\\geq y`\'\n\n    Testing a string enclosed in triple angle brackets::\n\n        sage: format(\'<<<identity_matrix\')\n        \'<<<identity_matrix\\n\'\n        sage: format(\'identity_matrix>>>\')\n        \'identity_matrix>>>\\n\'\n        sage: format(\'<<<identity_matrix>>>\')[:28]\n        \'Definition: identity_matrix(\'\n    '
sage: format?
Type:function
Base Class:<type 'function'>
String Form:<function format at 0xcc8a28>
Namespace:Interactive
File:/mnt/local/king/SAGE/broken/local/lib/python2.6/site-packages/sage/misc/sagedoc.py
Definition:format(s, embedded=False)
Docstring:
    <no docstring>

The problem is that the doc string contains <<<obj>>>, which means that it is attempted to find and insert documentation for obj (which does not exist).

Suggestion

Introduce a new directive noreplace, that avoids replacement of <<<obj>>>. Or perhaps a better suggestion: Catch the error, and do not replace if there is an error.

Second problem

If sage.misc.sagedoc.my_doc is applied to an object with a _sage_doc_ method then it uses it, without formatting. That means: No tex code is turned into ascii art, and embedding information is not stripped.

Suggestion

Consequently use sage.misc.sageinspect.sage_getdoc.

Apply attachment: trac11817_question_mark_using_sage_getdoc.2.patch

Depends on #11815

CC: @vbraun

Component: misc

Keywords: sage_getdoc format embedding

Author: Simon King

Reviewer: Volker Braun

Merged: sage-5.1.beta1

Issue created by migration from https://trac.sagemath.org/ticket/11817

[simon-king-jena]

comment:1

Actually I think it would be good to both have a "noreplace" directive and catching an error, when dealing with <<<obj>>>.

[simon-king-jena]

Attachment: trac11817_question_mark_using_sage_getdoc.patch.gz

Use sage_getdoc for interactively reading documentation. Improve docstring formatting

[simon-king-jena]

Author: Simon King

[simon-king-jena]

comment:3

With the attached patch, one can do:

sage: from sage.misc.sagedoc import format
sage: format?
Type:           function
Base Class:     <type 'function'>
String Form:    <function format at 0xc290c8>
Namespace:      Interactive
File:           /mnt/local/king/SAGE/debug/sage-4.7.2.alpha3-prerelease/local/lib/python2.6/site-packages/sage/misc/sagedoc.py
Definition:     format(s, embedded=False)
Docstring:
       Format Sage documentation ``s`` for viewing with IPython.

       This calls ``detex`` on ``s`` to convert LaTeX commands to plain
       text, unless the directive ``nodetex`` is given in the first line
       of the string.
...

Note that I introduced a noreplace directive, and used it on format's docstring.

Also, interactive doc read does not blindly use a custom _sage_doc_: It will always use sage_getdoc, which does preprocessing:

sage: r = 'some doc for a cython method\n`x \\geq y`'
sage: class Foo:
....:     def _sage_doc_(self):
....:         return r
....: 
sage: f = Foo()
sage: f?
Type:           instance
Base Class:     __main__.Foo
String Form:    <__main__.Foo instance at 0xb0ce18>
Namespace:      Interactive
File:           /mnt/local/king/SAGE/debug/sage-4.7.2.alpha3-prerelease/local/lib/python2.6/site-packages/IPython/FakeModule
.py   
Docstring:
    some doc for a cython method x >= y

Without the patch, the latex code would be shown.

If the noreplace directive is not used, but the to-be-replaced-with object can not be found, the replacement is simply skipped. Some new doctest related with directives and replacement:

sage: print format('File: bla.py (starting at line 1)\nnodetex, noreplace\n<<<identity_matrix>>>`\\not= 0`')
File: bla.py (starting at line 1)
<<<identity_matrix>>>`\not= 0`

sage: print format('<<<bla\n<<<bla>>>\n<<<identity_matrix>>>')
<<<bla <<<bla>>>
<BLANKLINE>
Definition: identity_matrix(ring, n=0, sparse=False)
<BLANKLINE>
   Return the n x n identity matrix over the given ring.
...

I didn't run all tests (only those of sage.misc.sagedoc)

[simon-king-jena]

comment:4

Is there a reviewer for a ticket on improving interactive documentation?

[vbraun]

Attachment: trac11817_question_mark_using_sage_getdoc.2.patch.gz

Rediffed for sage-5.0.rc0

[vbraun]

Reviewer: Volker Braun

[vbraun]

Description changed:

--- 
+++ 
@@ -33,3 +33,5 @@

 Consequently use `sage.misc.sageinspect.sage_getdoc`.

+
+Apply [attachment: trac11817_question_mark_using_sage_getdoc.2.patch](https://github.com/sagemath/sage-prod/files/10653686/trac11817_question_mark_using_sage_getdoc.2.patch.gz)

[vbraun]

comment:5

Patch doesn't apply to sage-5.0.rc0 because some docstring issue. Since its a trivial change I uploaded a rediffed patch.

[simon-king-jena]

comment:6

Thank you!

[jdemeyer]

Merged: sage-5.1.beta1

[fchapoton]

Description changed:

--- 
+++ 
@@ -9,12 +9,12 @@
 sage: format.__doc__
 '\n    Format Sage documentation ``s`` for viewing with IPython.\n\n    This calls ``detex`` on ``s`` to convert LaTeX commands to plain\n    text, and if ``s`` contains a string of the form "<<<obj>>>",\n    then it replaces it with the docstring for "obj".\n\n    INPUT:\n\n    - ``s`` - string\n    - ``embedded`` - boolean (optional, default False)\n\n    OUTPUT: string\n\n    Set ``embedded`` equal to True if formatting for use in the\n    notebook; this just gets passed as an argument to ``detex``.\n\n    EXAMPLES::\n\n        sage: from sage.misc.sagedoc import format\n        sage: identity_matrix(2).rook_vector.__doc__[115:184]\n        \'Let `A` be a general `m` by `n`\\n        (0,1)-matrix with `m \\\\le n`. \'\n        sage: format(identity_matrix(2).rook_vector.__doc__[115:184])\n        \'Let A be a general m by n\\n   (0,1)-matrix with m <= n.\\n\'\n\n    If the first line of the string is \'nodetex\', remove \'nodetex\' but\n    don\'t modify any TeX commands::\n    \n        sage: format("nodetex\\n`x \\\\geq y`")\n        \'\\n`x \\\\geq y`\'\n\n    Testing a string enclosed in triple angle brackets::\n\n        sage: format(\'<<<identity_matrix\')\n        \'<<<identity_matrix\\n\'\n        sage: format(\'identity_matrix>>>\')\n        \'identity_matrix>>>\\n\'\n        sage: format(\'<<<identity_matrix>>>\')[:28]\n        \'Definition: identity_matrix(\'\n    '
 sage: format?
-Type:      function
-Base Class:    <type 'function'>
-String Form:   <function format at 0xcc8a28>
-Namespace: Interactive
-File:      /mnt/local/king/SAGE/broken/local/lib/python2.6/site-packages/sage/misc/sagedoc.py
-Definition:    format(s, embedded=False)
+Type:function
+Base Class:<type 'function'>
+String Form:<function format at 0xcc8a28>
+Namespace:Interactive
+File:/mnt/local/king/SAGE/broken/local/lib/python2.6/site-packages/sage/misc/sagedoc.py
+Definition:format(s, embedded=False)
 Docstring:
     <no docstring>