Nested class name mangling can be wrong in case of double nesting

[hivert]

In the following class tree:

class Bla(UniqueRepresentation):
    class Bla1(UniqueRepresentation):
        class Bla11:
            Pass
    class Bla2:
        class Bla21:
            Pass

The names are set to

        sage: Bla.Bla1.__name__
        'Bla.Bla1'
        sage: Bla.Bla2.__name__
        'Bla.Bla2'
        sage: Bla.Bla2.Bla21.__name__
        'Bla.Bla2.Bla21'

But

        sage: Bla.Bla1.Bla11.__name__
        'Bla1.Bla11'

whereas one would expect 'Bla.Bla1.Bla11' This breaks a lot of doc in categories and in particular in functorial constructions.

Apply

• attachment: trac9107_nesting_nested_classes.patch
• attachment: trac_9107_fix_cross_reference.patch

Depends on #12808

CC: @simon-king-jena @zabrocki

Component: categories

Author: Simon King, Nicolas M. Thiéry

Branch: 8b14e05

Reviewer: Volker Braun, Florent Hivert, Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/9107

[simon-king-jena]

Dependencies: #12808

[simon-king-jena]

comment:2

I think we should make this depend on #12808, because it cythonises nested classes.

Here is my analysis:

1. In sage.misc.nested_class.modify_for_nested_pickling, only those attributes of a class are (recursively) renamed that are instances of type or of ClassType. However, ironically, instances of NestedClassMetaclass are ignored.
2. It is verified that the name of the to-be-changed class is identical with its name as an attribute of the calling class. But the renaming breaks the identity.

[simon-king-jena]

comment:3

I think the attached patch solves the problem. I get:

sage: class Bla(UniqueRepresentation):
....:     class Bla1(UniqueRepresentation):
....:         class Bla11:
....:             pass
....:     class Bla2:                   
....:         class Bla21:   
....:             pass
....:         
sage: Bla.Bla1.Bla11
<class __main__.Bla.Bla1.Bla11 at 0x46e7808>

The change is in modify_for_nested_pickle, which is called recursively. The idea is that the function should have an extra argument first_run, that is True by default. If the extra argument is False, then it is assumed that it is not applied for the first time.

Here: Since Bla.Bla1 is an instance of NestedClassMetaclass, modify_for_nested_pickle is called on Bla.Bla1.Bla11, resulting in Bla.Bla1.Bla11.__name__=='Bla1.Bla11'. However, since Bla is an instance of NestedClassMetaclass as well, the function is applied to Bla.Bla1 and thus recursively to Bla.Bla1.Bla11 another time.

Now, without my patch, in the second run, modify_for_nested_pickle would be confused by the fact that Bla.Bla1.__dict__ lists Bla.Bla1.Bla11 under the name Bla11, but Bla11.__name__=='Bla1.Bla11'. With my patch, modify_for_nested_pickle expects exactly that naming scheme, and is thus changing Bla.Bla1.Bla11.__name__ into "Bla.Bla1.Bla11".

Much BlaBla, but I think it works...

Potential problems

sage: module = sys.modules['__main__']
sage: getattr(module, 'Bla1.Bla11')                      
<class __main__.Bla.Bla1.Bla11 at 0x46e7808>
sage: getattr(module, 'Bla.Bla1.Bla11')
<class __main__.Bla.Bla1.Bla11 at 0x46e7808>

Hence, Bla.Bla1.Bla11 is listed in the module under two different names. If you think it is bad, then one could probably modify the function when first_run is false, such that the name given in the first run is erased from the module.

Moreover, the reviewer will likely find a speed regression, when excessively creating nested unique representations. But that's hardly realistic...

[simon-king-jena]

Author: Simon King

[simon-king-jena]

comment:4

Another problem: Source inspection does not work yet in the following example.

sage: cython_code = [
... "from sage.structure.unique_representation import UniqueRepresentation",
... "class A1(UniqueRepresentation):",
... "    class B1(UniqueRepresentation):",
... "        class C1: pass",
... "    class B2:",
... "        class C2: pass"]
sage: import os
sage: cython(os.linesep.join(cython_code))
sage: A1.B1.C1??          
Error getting source: class A1.B1.C1 has no attribute '__class__'
Type:       classobj
String Form:    _mnt_local_king__sage_temp_mpc622_6475_tmp_0_spyx_0.A1.B1.C1
Namespace:  Interactive
Loaded File:    /mnt/local/king/.sage/temp/mpc622/6475/spyx/_mnt_local_king__sage_temp_mpc622_6475_tmp_0_spyx/_mnt_local_king__sage_temp_mpc622_6475_tmp_0_spyx_0.so
Source File:    /mnt/local/king/.sage/temp/mpc622/6475/spyx/_mnt_local_king__sage_temp_mpc622_6475_tmp_0_spyx/_mnt_local_king__sage_temp_mpc622_6475_tmp_0_spyx_0.so

Even #11768 does not solve the problem.

Shall that be dealt with on a different ticket? Or in one go?

Probably on a different ticket, since I just find that even source inspection for A1 (which has a usual name) does not work...

[simon-king-jena]

comment:5

For the record: If #11791 is applied after this ticket, source inspection in the example above works (and is doctested).

[simon-king-jena]

comment:6

Is there a reviewer to fix name mangling of nested classes (needed in the category framework)?

[saliola]

comment:7

This patch also fixes an issue that came up in #8899 regarding documentation of nested classes not appearing in the reference manual.

See here for a description of the issue, see the thread on sage-combinat-devel.

See here for the confirmation that this works: #8899 comment:31

[vbraun]

comment:8

LGTM!

[vbraun]

Reviewer: Volker Braun

[jdemeyer]

comment:9

This causes trouble when building the documentation from scratch (i.e. after deleting 'devel/sage/doc/output`):

/usr/local/src/sage-5.5.rc1/local/lib/python2.7/site-packages/sage/categories/algebras_with_basis.py:docstring of sage.categories.algebras_with_basis.AlgebrasWithBasis.CartesianProducts.ParentMethods.one_from_cartesian_product_of_one_basis:3: WARNING: more than one target found for cross-reference u'one_basis': sage.combinat.sf.new_kschur.KBoundedSubspaceBases.ParentMethods.one_basis, sage.categories.algebras_with_basis.AlgebrasWithBasis.ParentMethods.one_basis, sage.combinat.ncsf_qsym.generic_basis_code.BasesOfQSymOrNCSF.ParentMethods.one_basis, sage.algebras.steenrod.steenrod_algebra.SteenrodAlgebra_generic.one_basis, sage.categories.examples.with_realizations.SubsetAlgebra.Fundamental.one_basis, sage.combinat.root_system.weyl_characters.WeightRing.one_basis, sage.categories.monoids.Monoids.Algebras.ParentMethods.one_basis, sage.categories.examples.hopf_algebras_with_basis.MyGroupAlgebra.one_basis, sage.categories.algebras_with_basis.AlgebrasWithBasis.TensorProducts.ParentMethods.one_basis, sage.algebras.affine_nil_temperley_lieb.AffineNilTemperleyLiebTypeA.one_basis, sage.categories.examples.algebras_with_basis.FreeAlgebra.one_basis, sage.combinat.symmetric_group_algebra.SymmetricGroupAlgebra_n.one_basis, sage.algebras.iwahori_hecke_algebra.IwahoriHeckeAlgebraT.one_basis, sage.algebras.group_algebra_new.GroupAlgebra.one_basis, sage.combinat.sf.sfa.SymmetricFunctionsBases.ParentMethods.one_basis, sage.combinat.root_system.weyl_characters.WeylCharacterRing.one_basis, sage.combinat.combinatorial_algebra.CombinatorialAlgebra.one_basis

[simon-king-jena]

comment:11

Jeroen, can you elaborate a bit what went wrong?

[simon-king-jena]

comment:12

Aha, now I see that the very long single line contains warnings about cross references that were not found. I'll try to identify them.

[simon-king-jena]

comment:13

Aha, here is an example:

The docstring of sage.categories.algebras_with_basis.AlgebrasWithBasis.CartesianProducts.ParentMethods.one_from_cartesian_product_of_one_basis is as follows:

            @cached_method   # todo: reinstate once #5843 is fixed
            def one_from_cartesian_product_of_one_basis(self):
                """
                Returns the one of this cartesian product of algebras, as per ``Monoids.ParentMethods.one``

                It is constructed as the cartesian product of the ones of the
                summands, using their :meth:`.one_basis` methods.

                This implementation does not require multiplication by
                scalars nor calling cartesian_product. This might help keeping
                things as lazy as possible upon initialization.
...

Could this simply be a misspelling? Note that it is written

:meth:`.one_basis`

but should certainly be

:meth:`one_basis`

If that's the case for the other warnings as well, then my patch would just uncover mistakes that happened earlier.

[jhpalmieri]

comment:14

The same issue arose in #13851 (see comment 10). I'm not sure why those dots are there, and I personally think they should be removed, but someone intentionally put them there.

[simon-king-jena]

comment:15

Replying to @jhpalmieri:

The same issue arose in #13851 (see comment 10). I'm not sure why those dots are there, and I personally think they should be removed, but someone intentionally put them there.

I think the dot is simply wrong - or is it ignored by Sphinx?

Actually here it is even worse. The text is documentation of a functorial construction, but refers to a parent method - that can't possibly work without an explicit reference to the method which must include the class which the method belongs to.

[simon-king-jena]

Attachment: trac_9107_fix_cross_reference.patch.gz

Fix a cross reference in some functorial construction

[simon-king-jena]

comment:16

Does the second patch fix the problem? I am now explicitly referring to the one_basis method of AlgebrasWithBasis.ParentMethods.

[hivert]

Changed reviewer from Volker Braun to Volker Braun, Florent Hivert

[hivert]

comment:17

Hi Simon,

I again hit this one compiling the doc. Your patches look all good to me, including the one problem.

Thanks,

Florent

[jdemeyer]

comment:18

Applying this patch causes the PDF docbuilder to hang after it's done building all documents. All documents are built but there are still 6 (I'm building with MAKE="make -j6") child processes which are stuck in the multiprocessing.Pool code. Interrupting those child processes simply causes new child processes to start which are then stuck again. It might be a bug in multiprocessing.Pool which is somehow triggered, I have no idea...

[jdemeyer]

comment:19

Perhaps an instance of #14323 (wild guess)?

[jdemeyer]

comment:20

No, #14323 doesn't help.

[simon-king-jena]

comment:21

Jeroen, does the problem persist with the new doc builder? I have just applied the two patches, and succeeded with export MAKE="make -j2" followed by make.

However, there is continuation by ... that needs fixing.

[simon-king-jena]

Attachment: trac9107_nesting_nested_classes.patch.gz

[simon-king-jena]

Description changed:

--- 
+++ 
@@ -28,4 +28,7 @@
 whereas one would expect `'Bla.Bla1.Bla11'`
 This breaks a lot of doc in categories and in particular in functorial constructions.

-Florent
+__Apply__
+
+- [attachment: trac9107_nesting_nested_classes.patch](https://github.com/sagemath/sage-prod/files/10649385/trac9107_nesting_nested_classes.patch.gz)
+- [attachment: trac_9107_fix_cross_reference.patch](https://github.com/sagemath/sage-prod/files/10649384/trac_9107_fix_cross_reference.patch.gz)

[simon-king-jena]

comment:22

Building the docs works for me, and the ... should be fixed now. Hence: Needs review!

Apply trac9107_nesting_nested_classes.patch trac_9107_fix_cross_reference.patch

[jdemeyer]

comment:23

Replying to @simon-king-jena:

Building the docs works for me

Also the PDF docs?

[jdemeyer]

comment:24

There is a problem with latex and the fact that the docbuilder hangs is a bug in the new docbuilder: #14626

! LaTeX Error: Too deeply nested.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...                                              

l.27819 \begin{Verbatim}[commandchars=\\\{\}]

? 
Implicit mode ON; LaTeX internals redefined
(/usr/share/texmf-texlive/tex/latex/ltxmisc/url.sty
(/usr/share/texmf-texlive/tex/latex/base/t1enc.def)
! Emergency stop.
 ...                                              

l.27819 \begin{Verbatim}[commandchars=\\\{\}]

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on categories.log.
)make[1]: *** [categories.pdf] Error 1

[simon-king-jena]

comment:26

Yes, I did not consider the pdf docs.

If I understand correctly, we have two problems. The first problem is that with this patch, LaTeX is produced that can not be processed because it is two deeply nested. The second problem is independent, namely if latex fails, then the docbuilder hangs.

Do you have any clue what object is being processed when things hang?

[jdemeyer]

comment:27

Replying to @simon-king-jena:

The second problem is independent, namely if latex fails, then the docbuilder hangs.

Which is #14626 and indeed has nothing to do with this ticket.

Do you have any clue what object is being processed when things hang?

Not yet, I will reproduce the .tex file and then it should be clear.

[jdemeyer]

comment:28

Offending .tex file: http://boxen.math.washington.edu/home/jdemeyer/badlatex/categories.tex

The relevant lines are

\begin{fulllineitems}
\phantomsection\label{sage/categories/sets_cat:sage.categories.sets_cat.Sets.WithRealizations.ParentMethods}\pysigline{\strong{class }\bfcode{ParentMethods}}~\index{Sets.WithRealizations.ParentMethods.Realizations (class in sage.categories.sets\_cat)}

\begin{fulllineitems}
\phantomsection\label{sage/categories/sets_cat:sage.categories.sets_cat.Sets.WithRealizations.ParentMethods.Realizations}\pysiglinewithargsret{\strong{class }\bfcode{Realizations}}{\emph{parent\_with\_realization}}{}
Bases: {\hyperref[sage/categories/realizations:sage.categories.realizations.Category_realization_of_parent]{\code{sage.categories.realizations.Category\_realization\_of\_parent}}}

TESTS:

\begin{Verbatim}[commandchars=\\\{\}]
\PYG{g+gp}{sage: }\PYG{k+kn}{from} \PYG{n+nn}{sage.categories.realizations} \PYG{k+kn}{import} \PYG{n}{Category\PYGZus{}realization\PYGZus{}of\PYGZus{}parent}
\PYG{g+gp}{sage: }\PYG{n}{A} \PYG{o}{=} \PYG{n}{Sets}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{WithRealizations}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{example}\PYG{p}{(}\PYG{p}{)}\PYG{p}{;} \PYG{n}{A}
\PYG{g+go}{The subset algebra of \PYGZob{}1, 2, 3\PYGZcb{} over Rational Field}
\PYG{g+gp}{sage: }\PYG{n}{C} \PYG{o}{=} \PYG{n}{A}\PYG{o}{.}\PYG{n}{Realizations}\PYG{p}{(}\PYG{p}{)}\PYG{p}{;} \PYG{n}{C}
\PYG{g+go}{Category of realizations of The subset algebra of \PYGZob{}1, 2, 3\PYGZcb{} over Rational Field}
\PYG{g+gp}{sage: }\PYG{n+nb}{isinstance}\PYG{p}{(}\PYG{n}{C}\PYG{p}{,} \PYG{n}{Category\PYGZus{}realization\PYGZus{}of\PYGZus{}parent}\PYG{p}{)}
\PYG{g+go}{True}
\PYG{g+gp}{sage: }\PYG{n}{C}\PYG{o}{.}\PYG{n}{parent\PYGZus{}with\PYGZus{}realization}
\PYG{g+go}{The subset algebra of \PYGZob{}1, 2, 3\PYGZcb{} over Rational Field}
\PYG{g+gp}{sage: }\PYG{n}{TestSuite}\PYG{p}{(}\PYG{n}{C}\PYG{p}{)}\PYG{o}{.}\PYG{n}{run}\PYG{p}{(}\PYG{p}{)}
\end{Verbatim}
\index{super\_categories() (sage.categories.sets\_cat.Sets.WithRealizations.ParentMethods.Realizations method)}

\begin{fulllineitems}
\phantomsection\label{sage/categories/sets_cat:sage.categories.sets_cat.Sets.WithRealizations.ParentMethods.Realizations.super_categories}\pysiglinewithargsret{\bfcode{super\_categories}}{}{}
EXAMPLES:

\begin{Verbatim}[commandchars=\\\{\}]   %% PROBLEM IS THIS LINE %%
\PYG{g+gp}{sage: }\PYG{n}{A} \PYG{o}{=} \PYG{n}{Sets}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{WithRealizations}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{example}\PYG{p}{(}\PYG{p}{)}\PYG{p}{;} \PYG{n}{A}
\PYG{g+go}{The subset algebra of \PYGZob{}1, 2, 3\PYGZcb{} over Rational Field}
\PYG{g+gp}{sage: }\PYG{n}{A}\PYG{o}{.}\PYG{n}{Realizations}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{super\PYGZus{}categories}\PYG{p}{(}\PYG{p}{)}
\PYG{g+go}{[Category of realizations of sets]}
\end{Verbatim}

\end{fulllineitems}

\end{fulllineitems}

\index{facade\_for() (sage.categories.sets\_cat.Sets.WithRealizations.ParentMethods method)}

[jdemeyer]

comment:29

before this patch (good):

\phantomsection\label{sage/categories/sets_cat:sage.categories.sets_cat.Sets.WithRealizations.ParentMethods}\pysigline{\bfcode{ParentMethods}}
alias of \code{WithRealizations.ParentMethods}

after this patch (bad):

\phantomsection\label{sage/categories/sets_cat:sage.categories.sets_cat.Sets.WithRealizations.ParentMethods.Realizations}\pysiglinewithargsret{\strong{class }\bfcode{Realizations}}{\emph{parent\_with\_realization}}{}
Bases: {\hyperref[sage/categories/realizations:sage.categories.realizations.Category_realization_of_parent]{\code{sage.categories.realizations.Category\_realization\_of\_parent}}}

[simon-king-jena]

comment:30

Replying to @jdemeyer:

before this patch (good):

\phantomsection\label{sage/categories/sets_cat:sage.categories.sets_cat.Sets.WithRealizations.ParentMethods}\pysigline{\bfcode{ParentMethods}}
alias of \code{WithRealizations.ParentMethods}

after this patch (bad):

\phantomsection\label{sage/categories/sets_cat:sage.categories.sets_cat.Sets.WithRealizations.ParentMethods.Realizations}\pysiglinewithargsret{\strong{class }\bfcode{Realizations}}{\emph{parent\_with\_realization}}{}
Bases: {\hyperref[sage/categories/realizations:sage.categories.realizations.Category_realization_of_parent]{\code{sage.categories.realizations.Category\_realization\_of\_parent}}}

Three questions:

1. Why is it bad? I don't see why latex should have a problem with it.

2. Isn't the "good" output without my patch just plain wrong? After all, we do have

   sage: sage.categories.sets_cat.Sets.WithRealizations.ParentMethods.Realizations.__bases__
   (sage.categories.realizations.Category_realization_of_parent,)

   and also sage.categories.sets_cat.Sets.WithRealizations.ParentMethods.Realizations is certainly not simply an alias of WithRealizations.ParentMethods.

3. Can you also point me to the code that created the latex output?

[jhpalmieri]

comment:31

When I build the pdf docs, it works. On what machine do you see the failure? If it's on sage.math, it might have to do with the fact that the LaTeX installation is quite old...

Edit: maybe I'm seeing the failure now. Never mind.

[simon-king-jena]

comment:32

OK, I see it, too.

../../sage -docbuild reference pdf
...
Output written on tensor.pdf (24 pages, 144532 bytes).
Transcript written on tensor.log.
Build finished.  The built documents can be found in /home/simon/SAGE/prerelease/sage-5.9.rc0/devel/sage/doc/output/pdf/en/reference/tensor

and then it hangs.

Nevertheless, I have no clue what is happening here. See my three questions in comment:30.

[jdemeyer]

comment:33

Replying to @simon-king-jena:

1. Why is it bad?

I just used "bad" because latex doesn't compile it correctly.

3. Can you also point me to the code that created the latex output?

I guess that's Sphinx, but I don't know much about it.

[simon-king-jena]

comment:34

Replying to @jdemeyer:

Replying to @simon-king-jena:

1. Why is it bad?

I just used "bad" because latex doesn't compile it correctly.

That was my question: Why does latex not compile it correctly?

And we should keep in mind that the old output has simply been wrong.

[jhpalmieri]

comment:35

I think that the first line in the LaTeX error message is correct:

! LaTeX Error: Too deeply nested.

I think that there are too many levels of nesting of lists (from the fulllineitems environment). If I comment out the Verbatim environment that it's complaining about, I don't get an error message any more.

[simon-king-jena]

comment:36

Replying to @jhpalmieri:

I think that the first line in the LaTeX error message is correct:

! LaTeX Error: Too deeply nested.

I think that there are too many levels of nesting of lists (from the fulllineitems environment). If I comment out the Verbatim environment that it's complaining about, I don't get an error message any more.

Please, where is the nesting? I suppose by "comment out the Verbatim environment that it's complaining about", you mean one of two Verbatim environments that were cited in comment:28.

The first is

\begin{Verbatim}[commandchars=\\\{\}]
\PYG{g+gp}{sage: }\PYG{k+kn}{from} \PYG{n+nn}{sage.categories.realizations} \PYG{k+kn}{import} \PYG{n}{Category\PYGZus{}realization\PYGZus{}of\PYGZus{}parent}
\PYG{g+gp}{sage: }\PYG{n}{A} \PYG{o}{=} \PYG{n}{Sets}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{WithRealizations}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{example}\PYG{p}{(}\PYG{p}{)}\PYG{p}{;} \PYG{n}{A}
\PYG{g+go}{The subset algebra of \PYGZob{}1, 2, 3\PYGZcb{} over Rational Field}
\PYG{g+gp}{sage: }\PYG{n}{C} \PYG{o}{=} \PYG{n}{A}\PYG{o}{.}\PYG{n}{Realizations}\PYG{p}{(}\PYG{p}{)}\PYG{p}{;} \PYG{n}{C}
\PYG{g+go}{Category of realizations of The subset algebra of \PYGZob{}1, 2, 3\PYGZcb{} over Rational Field}
\PYG{g+gp}{sage: }\PYG{n+nb}{isinstance}\PYG{p}{(}\PYG{n}{C}\PYG{p}{,} \PYG{n}{Category\PYGZus{}realization\PYGZus{}of\PYGZus{}parent}\PYG{p}{)}
\PYG{g+go}{True}
\PYG{g+gp}{sage: }\PYG{n}{C}\PYG{o}{.}\PYG{n}{parent\PYGZus{}with\PYGZus{}realization}
\PYG{g+go}{The subset algebra of \PYGZob{}1, 2, 3\PYGZcb{} over Rational Field}
\PYG{g+gp}{sage: }\PYG{n}{TestSuite}\PYG{p}{(}\PYG{n}{C}\PYG{p}{)}\PYG{o}{.}\PYG{n}{run}\PYG{p}{(}\PYG{p}{)}
\end{Verbatim}

the second is

\begin{Verbatim}[commandchars=\\\{\}]   %% PROBLEM IS THIS LINE %%
\PYG{g+gp}{sage: }\PYG{n}{A} \PYG{o}{=} \PYG{n}{Sets}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{WithRealizations}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{example}\PYG{p}{(}\PYG{p}{)}\PYG{p}{;} \PYG{n}{A}
\PYG{g+go}{The subset algebra of \PYGZob{}1, 2, 3\PYGZcb{} over Rational Field}
\PYG{g+gp}{sage: }\PYG{n}{A}\PYG{o}{.}\PYG{n}{Realizations}\PYG{p}{(}\PYG{p}{)}\PYG{o}{.}\PYG{n}{super\PYGZus{}categories}\PYG{p}{(}\PYG{p}{)}
\PYG{g+go}{[Category of realizations of sets]}
\end{Verbatim}

I suppose %% PROBLEM IS THIS LINE %% in the second environment was Jeroen's addition.

So, what is "too deeply nested"? I can't believe that such a short piece of text has even enough characters to nest too deeply for latex!

[jhpalmieri]

comment:37

If I take the file categories.tex in SAGE_ROOT/devel/sage/doc/output/latex/en/reference/categories/ and truncate it just before the line starting \index{facade\_for() ..., then I need to add in a few lines of the form

\end{fulllineitems}

to get it to compile (after I comment out the last Verbatim block before the line \index{facade\_for() ...). So there are several fulllineitems environments nested within each other. Maybe too many, and maybe that's the problem. That's my current guess.

[tscrim]

comment:39

Hey Nicolas and Simon,

The problem comes from the fact that there is a 4 level deep class nesting with a method (which is 5 levels deep) in the Sets.WithRealizations.ParentMethods.Realizations.super_categories. I've tried moving this subclass into a separate class, and this solves the pdf build issue but introduces some doctesting errors. I don't think there is a to extend the nesting level since that is a latex thing, nor do I think we should try since 4 nested classes is a lot IMO. I'm guessing beforehand because of the improper naming, latex did the environments differently...?

Anyways the fix for the pdf build is to remove a level (or two) of class nesting.

Best,

Travis

Edit: Here are the errors I get when I move Sets.WithRealizations out as a separate class and then assign it into Sets:

sage -t ../categories/sets_cat.py
**********************************************************************
File "../categories/sets_cat.py", line 1408, in sage.categories.sets_cat.ParentMethodsForWithRealizations.realizations
Failed example:
    A.realizations()
Expected:
    [The subset algebra of {1, 2, 3} over Rational Field in the Fundamental basis, The subset algebra of {1, 2, 3} over Rational Field in the In basis, The subset algebra of {1, 2, 3} over Rational Field in the Out basis]
Got:
    [The subset algebra of {1, 2, 3} over Rational Field in the Fundamental basis, The subset algebra of {1, 2, 3} over Rational Field in the In basis, The subset algebra of {1, 2, 3} over Rational Field in the Out basis, The subset algebra of {1, 2, 3} over Rational Field in the realization Blah]
**********************************************************************
File "../categories/sets_cat.py", line 1428, in sage.categories.sets_cat.ParentMethodsForWithRealizations.facade_for
Failed example:
    A.facade_for()
Expected:
    [The subset algebra of {1, 2, 3} over Rational Field in the Fundamental basis, The subset algebra of {1, 2, 3} over Rational Field in the In basis, The subset algebra of {1, 2, 3} over Rational Field in the Out basis]
Got:
    [The subset algebra of {1, 2, 3} over Rational Field in the Fundamental basis, The subset algebra of {1, 2, 3} over Rational Field in the In basis, The subset algebra of {1, 2, 3} over Rational Field in the Out basis, The subset algebra of {1, 2, 3} over Rational Field in the realization Blah]
**********************************************************************
2 items had failures:
   1 of   8 in sage.categories.sets_cat.ParentMethodsForWithRealizations.facade_for
   1 of   3 in sage.categories.sets_cat.ParentMethodsForWithRealizations.realizations
    [241 tests, 2 failures, 0.76 s]
----------------------------------------------------------------------
sage -t ../categories/sets_cat.py  # 2 doctests failed
----------------------------------------------------------------------

Any ideas why moving the class out of the nesting doesn't work?

[simon-king-jena]

comment:40

Replying to @tscrim:

I don't think there is a to extend the nesting level since that is a latex thing,

Shame on LaTeX!

Anyways the fix for the pdf build is to remove a level (or two) of class nesting.

What exactly are we talking about? Sets.WithRealizations.ParentMethods.Realizations?

Interestingly, there is the comment

# Do we really want this feature?

So, can we do without this feature? Nicolas?

[tscrim]

comment:41

Replying to @simon-king-jena:

Replying to @tscrim:

Anyways the fix for the pdf build is to remove a level (or two) of class nesting.

What exactly are we talking about? Sets.WithRealizations.ParentMethods.Realizations?

Yes. Removing a level of nesting allowed the pdf for categories to build for me.

[nthiery]

comment:42

Thanks much Travis for investigating!

I agree that there should be a recommendation for not nesting classes too deep, for the sake of readability. But having a hard arbitrary limit -- especially that small -- is annoying. Shame on LaTeX. Of course, one can always spin off a subtree of nested classes into a separate tree, but there are cases where one has a deep tree with very few lines and no natural splitting point. For example, #10963 introduces

    DistributiveMagmasAndAdditiveMagmas.AdditiveAssociative.AdditiveCommutative.AdditiveUnital.AdditiveInverse

Hmm. Altogether, I would call this a LaTeX arbitrary hard limitation. Luckily there seems to be an easy solution to increase this limitation to something large enough to cover our current use cases, namely to use the package enumitem [1]. By itself, it brings the nesting level to 6, and we could even increase it further (10 should be really safe) using \setlistdepth{9}.

I have attached the little latex file I used for testing.

What do you think? Shall we add enumitems to the list of latex packages loaded by Sphinx? Is this standard enough, or shall we add enumitem.sty to the Sage distribution?

Cheers, Nicolas

[1] http://stackoverflow.com/questions/1935952/maximum-nesting-level-of-lists-in-latex

[nthiery]

Attachment: test-deep-itemize-nesting.tex.gz

[jdemeyer]

comment:43

enumitem.sty looks pretty standard, so I'd say it's fine to use it.

[simon-king-jena]

comment:44

Replying to @jdemeyer:

enumitem.sty looks pretty standard, so I'd say it's fine to use it.

... which means there should be a separate ticket for adding it?

[jdemeyer]

comment:45

Replying to @simon-king-jena:

... which means there should be a separate ticket for adding it?

Adding an \usepackage{} somewhere (don't ask me where) should be trivial enough that it can be done on this ticket.

[simon-king-jena]

comment:46

Replying to @jdemeyer:

Replying to @simon-king-jena:

... which means there should be a separate ticket for adding it?

Adding an \usepackage{} somewhere (don't ask me where) should be trivial enough that it can be done on this ticket.

So, whom do we ask?