Closed tscrim closed 7 years ago
Oh, I forgot about cafb215ba2b808e: what's the rationale for replacing .map_coefficients by what looks like hand written equivalent?
Oops, failing tests. Probably a triviality. Investigating.
Branch pushed to git repo; I updated commit sha1. New commits:
06a299e | 22632: fix typo in attribute name |
It comes from the same commit :-) Just a typo in _monomial_cofficients
. Fixed.
New commits:
06a299e | 22632: fix typo in attribute name |
sage -t src/sage/homology/simplicial_complex.py # 1 doctest failed
sage -t src/sage/algebras/iwahori_hecke_algebra.py # 18 doctests failed
sage -t src/sage/calculus/calculus.py # 1 doctest failed
sage -t src/sage/categories/sets_cat.py # 1 doctest failed
sage -t src/sage/categories/hopf_algebras_with_basis.py # 1 doctest failed
sage -t src/sage/categories/algebras_with_basis.py # 1 doctest failed
sage -t src/sage/combinat/free_module.py # 1 doctest failed
Two types of failures:
...
return type(self)(F, {k: c._divide_if_possible(x)
File "sage/structure/element.pyx", line 459, in sage.structure.element.Element.__getattr__ (/opt/sage/src/build/cythonized/sage/structure/element.c:4256)
return self.getattr_from_category(name)
File "sage/structure/element.pyx", line 472, in sage.structure.element.Element.getattr_from_category (/opt/sage/src/build/cythonized/sage/structure/element.c:4365)
return getattr_from_other_class(self, cls, name)
File "sage/structure/misc.pyx", line 300, in sage.structure.misc.getattr_from_other_class (/opt/sage/src/build/cythonized/sage/structure/misc.c:1933)
raise dummy_attribute_error
AttributeError: 'sage.rings.polynomial.laurent_polynomial.LaurentPolynomial_univariate' object has no attribute '_divide_if_possible'
and trivial one about the element_class name. Working on the latter.
The former probably comes from a parent whose categories are not initialized.
Replying to @nthiery:
The former probably comes from a parent whose categories are not initialized.
The problem comes from the fact that Laurent polynomials are not in Euclidean domains. There are also a number of natural rings that are not Euclidean domains, but have a quo_rem
that we could at least try this with (e.g., Z[x]). So I'm thinking of demoting it back down from the category.
I just found out the same :-) Yeah, either demoting it back down, or lifting it up to Rings(). After all, it's a private method. So if it fails because it requires another method which is not implemented, that's ok. At least it will be easier to discover than if it's hidden in some random Python module. It's consistent with FreeModule's requirement of taking a ring as input.
For the other failure: it stems from the fact that the created element class's name is set to reflect that it's the element class of a given parent:
sage: A = Algebras(QQ).WithBasis().example()
sage: A.element_class
<class 'sage.combinat.free_module.FreeAlgebra_with_category.element_class'>
However the work is half baked, since the module is not set
accordingly: there is no class
sage.combinat.free_module.FreeAlgebra
; the module part comes from
CombinatorialFreeModuleElement.__module__
(I am running this example
without the ticket). With the ...Element
being moved, the module
part gets changed to sage.modules.with_basis.indexed_element
.
Two options:
I just implemented the latter to check that it worked. The end result is more consistent for the user; here we get:
<class 'sage.categories.examples.algebras_with_basis.FreeAlgebra_with_category.element_class'>
However changing __module__
might possibly break some introspection
stuff (I checked A.element_class??
, and that was fine). It will also
require updating other doctests.
What do you think?
Branch pushed to git repo; I updated commit sha1. New commits:
c889c4e | 22632: Fix the module part in dynamically created element classes, for consistency |
Replying to @nthiery:
I just found out the same :-) Yeah, either demoting it back down, or lifting it up to Rings(). After all, it's a private method. So if it fails because it requires another method which is not implemented, that's ok. At least it will be easier to discover than if it's hidden in some random Python module. It's consistent with FreeModule's requirement of taking a ring as input.
I think lifting it up to the category of Rings
works for now.
Replying to @nthiery:
For the other failure: it stems from the fact that the created element class's name is set to reflect that it's the element class of a given ... Two options:
- Not care and just update the doctests
Improve the output of the element class by setting the module of the parent as module.
What do you think?
This seems to be a can of worms. :/ Although I would rather have the latter as well since it is a more proper fix. Since you've done it, we might as well keep it.
Ok; tests running here. Let's keep it only if there are just trivial doctests updates.
Branch pushed to git repo; I updated commit sha1. New commits:
b11d26e | 22632: trivial indentation fix |
All doctests updates were trivial. Being tired, I may have missed a few. Please rerun the tests, and if all goes well, or there just are some more trivial doctest updates, you can set a positive review on my behalf.
Cheers
Tests pass for me too.
Thank you for the review.
My pleasure! I am glad this is done.
PDF docs dont build
Compiling the doc here ...
Branch pushed to git repo; I updated commit sha1. New commits:
20cd0da | 22632: fixed ReST typo preventing the pdf doc compilation |
Arg, the exact same _mul_
mistake has appeared a couple days ago in some other ticket.
I'll set back a positive review when the pdf doc compilation will be finished, in case there wouldbe something else.
Thanks Volker for reporting ...
The only other error I get is:
[docpdf] l.42 \addto
[docpdf] \extrasmagyar{\def\pageautorefname{page}}
And I assume this is related to an outdate latex install on my machine:
[docpdf] Package sphinx Warning:
[docpdf] ******** ERROR !! PLEASE UPDATE titlesec.sty !!********
[docpdf] ******** THIS VERSION SWALLOWS SECTION NUMBERS.********.
So, putting this back to positive review. Travis, can you double check things compile on your machine?
pdf doc built for me too.
Changed branch from public/combinat/cythonize_CFM_element-22632 to 20cd0da
Question: what exactly is the purpose of the condition involving is_extension_type()
in __make_element_class__
?
I am trying to understand how/why is_extension_type
is used in Sage. In other places in parent.pyx
, it is to determine whether __class__
can be assigned to. But in __make_element_class__
, it's not so clear to me.
(I know it's not directly related to this ticket, but git blame
brought me here since this ticket made changes to __make_element_class__
)
Hi Jeroen!
In __make_element_class__
, we need to decide how elements shall inherit from categories: either by subclassing P.Element
(or the relevant class), or by using the getattr
trick. The current rule is to subclass if P.Element
is a Python class, and use getattr
if P.Element
is an extension type. is_extension_type
is used to do the test.
Thanks. That answers the factual question ("how is it done?") but not really the reason.
There are many differences between Python and Cython classes and I'm trying to understand which property matters here. I am asking because I tried to always use the dynamic class (using inherit=True
in __make_element_class__
even for Cython element classes) and there is not much that breaks.
These are the main differences between Python and Cython classes that I can think of:
The fact that __class__
can be assigned to for Python classes (I don't think this is the reason here)
The fact that Python classes have a __dict__
(this is no longer true: Python classes with __slots__
don't have a __dict__
by default and Cython classes do support __dict__
if you declare it as attribute)
The fact that Python classes always support multiple inheritance, which may or may not work for Cython classes
Pickling (although Cython 0.26 does support pickling of Cython classes)
Various performance differences
...
Replying to @jdemeyer:
I am asking because I tried to always use the dynamic class (using
inherit=True
in__make_element_class__
even for Cython element classes) and there is not much that breaks.
To elaborate on this: after making this change
diff --git a/src/sage/structure/parent.pyx b/src/sage/structure/parent.pyx
index a387136..ea7c60c 100644
--- a/src/sage/structure/parent.pyx
+++ b/src/sage/structure/parent.pyx
@@ -571,7 +571,7 @@ cdef class Parent(sage.structure.category_object.CategoryObject):
# By default, don't fiddle with extension types yet; inheritance from
# categories will probably be achieved in a different way
if inherit is None:
- inherit = not is_extension_type(cls)
+ inherit = True
if inherit:
if name is None:
name = "%s_with_category"%cls.__name__
I ran all doctests in rings
, categories
and structure
and the only non-trivial doctest failures were involving enumerated sets (src/sage/categories/examples/infinite_enumerated_sets.py
, src/sage/categories/sets_cat.py
, src/sage/categories/enumerated_sets.py
)
Hi Jeroen,
Thanks for running this interesting experiment!
I indeed would have been expecting things not too break too much. Systematically using dynamic classes would certainly simplify Sage's infrastructure. The only potential reason for not doing it is performance (speed and memory footprint). At the time we set this up, there was some consensual claim that systematically using a dynamic class would induce a performance loss for low granularity objects (e.g. finite field elements). Hence the above rule of thumb I implemented for deciding when to do what.
However I don't remember anyone doing some serious benchmarking to actually assess the validity of that claim. That would be very welcome! And this hints again at having some speed-regression test infrastructure for Sage.
Cheers, Nicolas
PS: I could imagine code breaking in situations like:
cdef class MyElement:
....
cdef foo(MyElement x):
x.bar()
where foo
would be called on an instance of a dynamic subclass of MyElement
, and foo
be provided by some category.
Apparently we have no instance of that idiom in Sage :-)
Comments on this actual ticket: contrary to most element classes which are implemented in Cython, it seems that CombinatorialFreeModule_with_category.element_class
actually does do the inheritance from the category in the "normal" way, using dynamic_class()
, just like Python classes. So this is a good testcase for #23435.
In the light of #23435, is it important that CombinatorialFreeModule_with_category.element_class
has a __dict__
? In other words, is it important to support the following (example taken from src/doc/en/thematic_tutorials/tutorial-objects-and-classes.rst
):
Some particular actions modify the data structure of ``el``::
sage: el.rename("bla")
sage: el
bla
.. note::
The class is stored in a particular attribute called ``__class__``,
and the normal attributes are stored in a dictionary called ``__dict__``::
sage: F = CombinatorialFreeModule(QQ, Permutations())
sage: el = 3*F([1,3,2])+ F([1,2,3])
sage: el.rename("foo")
sage: el.blah = 42
sage: el.__class__
<class 'sage.combinat.free_module.CombinatorialFreeModule_with_category.element_class'>
sage: el.__dict__
{'__custom_name': 'foo',
'blah': 42}
Another question about this ticket: why are the arithmetic methods of IndexedFreeModuleElement
implemented as cdef
instead of cpdef
?
cdef _add_(self, other):
Every other class (except for IndexedFreeModuleElement
and LieAlgebraElement
) implements those as cpdef _add_
.
This is confusing because
sage: F = CombinatorialFreeModule(QQ, Permutations())
sage: el = 3*F([1,3,2])+ F([1,2,3])
sage: el._add_(el)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-31-91def7298b70> in <module>()
----> 1 el._add_(el)
TypeError: 'NotImplementedType' object is not callable
It also prevents a Python class inheriting from IndexedFreeModuleElement
to override this _add_
.
Replying to @jdemeyer:
Another question about this ticket: why are the arithmetic methods of
IndexedFreeModuleElement
implemented ascdef
instead ofcpdef
?cdef _add_(self, other):
Every other class (except for
IndexedFreeModuleElement
andLieAlgebraElement
) implements those ascpdef _add_
.
That is not quite true. They are cdef
in Element
, and so I followed suit. I see now in the doc for Element
, that it says it should be cpdef
. Is it because Element
is special in that they are essentially acting as pure virtual methods that they aren't cpdef
?
Follow-up: #23440
Replying to @tscrim:
That is not quite true. They are
cdef
inElement
, and so I followed suit. I see now in the doc forElement
, that it says it should becpdef
. Is it becauseElement
is special in that they are essentially acting as pure virtual methods that they aren'tcpdef
?
Yes, the Element
base class is special. The arithmetic methods of Element
do indeed act as "pure virtual" methods. I implemented them that way to allow fast calling in the Cython world while at the same time not existing in the Python world.
We move
CombintorialFreeModuleElement
to its own classIndexedFreeModuleElement
and cythonize it for (future) speed gains.CC: @sagetrac-sage-combinat @nthiery @jdemeyer @fchapoton
Component: performance
Keywords: days85
Author: Travis Scrimshaw
Branch:
20cd0da
Reviewer: Nicolas M. Thiéry
Issue created by migration from https://trac.sagemath.org/ticket/22632