Make parent/element classes independent of base rings

[simon-king-jena]

At #11900 and sage-combinat-devel, as well as in some comments in sage/categories/category.py, the idea was discussed to make, for example, Algebras(GF(3)).parent_class==Algebras(GF(5)).parent_class - hence, make the parent/element classes as independent from the base of a category as possible.

This is implemented in this patch by introducing an abstract class CategoryWithParameters which uses pickling by "weak construction" for its element and parent classes. Now:

• For a join category, the parent/element class depend only on the parent/element class of its super categories.
• For a Category_over_base (e.g. Modules, Algebras, Schemes, ...), the parent/element class depend only on the category of the base.
• For a bimodule, the parent/element class depend only on the category of the left and right bases.

In the process, this patch also:

• Adds a method Category._make_named_class providing a unified way to create parent and element classes (and later on morphism classes)
• Extends the interface of dynamic_class to customize caching and pickling
• Rename the experimental class IdempotentSemigroups.ElementMethods and remove its super class, and discards unused code there.

Apply

• attachment: trac11935_weak_pickling_by_construction-nt.patch
• attachment: trac_11935-weak_pickling_by_construction-review-ts.patch

Depends on #9138 Depends on #11900 Depends on #11943 Depends on #12875 Depends on #12876 Depends on #12877

CC: @jdemeyer @sagetrac-sage-combinat

Component: categories

Keywords: parent class, element class

Author: Simon King

Reviewer: Nicolas M. Thiéry, Travis Scrimshaw

Merged: sage-5.11.beta1

Issue created by migration from https://trac.sagemath.org/ticket/11935

[nthiery]

comment:80

Just a try to have the patchbot try this ticket without #12876!

[nthiery]

Changed dependencies from #9138, #11900, #11943, #12875, #12876, #12877 to #9138, #11900, #11943, #12875, #12877

[nthiery]

Description changed:

--- 
+++ 
@@ -2,15 +2,25 @@

 This is implemented in this patch by introducing an abstract class
 CategoryWithParameters which uses pickling by "weak construction" for
-its element and parent classes. In the process, this patch also:
+its element and parent classes. Now:
+
+- For a join category, the parent/element class depend only on the
+  parent/element class of its super categories.
+- For a Category_over_base (e.g. Modules, Algebras, Schemes, ...), the
+  parent/element class depend only on the category of the base.
+- For a bimodule, the parent/element class depend only on the category
+  of the left and right bases.
+
+In the process, this patch also:

 - Adds a method Category._make_named_class providing a unified way to
   create parent and element classes (and later on morphism classes)
 - Extends the interface of dynamic_class to customize caching and pickling
-
+- Rename the experimental class IdempotentSemigroups.ElementMethods
+  and remove its super class, and discards unused code there.

 __Apply__

-* [attachment: trac11935_weak_pickling_by_construction_rel11943-nt.patch](https://github.com/sagemath/sage-prod/files/10653895/trac11935_weak_pickling_by_construction_rel11943-nt.patch.gz)
+* [attachment: trac11935_weak_pickling_by_construction-nt.patch](https://github.com/sagemath/sage-prod/files/10653897/trac11935_weak_pickling_by_construction-nt.patch.gz)

[nthiery]

comment:82

Attachment: trac11935_weak_pickling_by_construction-nt.patch.gz

Hi Simon,

I just went through the history of this ticket. At this point, I feel it's a reasonable choice to have the parent/element classe of a category over base depend only on the base.

The patch I just uploaded includes:

• The latest version of trac11935_weak_pickling_by_construction_rel11943-nt.patch in the Sage-Combinat queue (which was that from trac + some rebase + little improvements)
• trac11935_share_on_base_category.patch
• Doctests for the _make_named_class_key methods that were missing it
• Some doc minor improvements

With this patch applied on top of #12876, all long test passed on my machine, and I still get the desired speedup for our standard elliptic curve benchmark:

sage: %time L = EllipticCurve('960d1').prove_BSD()
CPU times: user 2.98 s, sys: 0.08 s, total: 3.06 s
Wall time: 3.17 s

sage: %time L = EllipticCurve('960d1').prove_BSD()
CPU times: user 2.06 s, sys: 0.04 s, total: 2.10 s
Wall time: 2.20 s

From my point of view, it's good to go. To completely finish the review I just need someone to proofread the doctests for the _make_named_class_key methods. And you may want to glance through the patch.

Note: I have decided to fold the patches together because anyway trac11935_weak_pickling_by_construction_rel11943-nt.patch was changed in the mean time. Also, since this has been a long time, I guess its easier to glance through and see the big picture. And it's not so long anyway.

Cheers, Nicolas

[tscrim]

comment:83

Hey Nicolas and Simon,

I've looked over the documentation. I've uploaded a small review patch (also to to the combinat queue) which makes some minor tweaks to the doc and adds a few more tests to CategoryWithParameters._make_named_class(). If you're happy with my changes, then you can set this to positive review.

Best,

Travis

[tscrim]

Reviewer: Nicolas Thiery, Travis Scrimshaw

[tscrim]

Description changed:

--- 
+++ 
@@ -23,4 +23,4 @@
 __Apply__

 * [attachment: trac11935_weak_pickling_by_construction-nt.patch](https://github.com/sagemath/sage-prod/files/10653897/trac11935_weak_pickling_by_construction-nt.patch.gz)
-
+* [attachment: trac_11935-weak_pickling_by_construction-review-ts.patch](https://github.com/sagemath/sage-prod/files/10653898/trac_11935-weak_pickling_by_construction-review-ts.patch.gz)

[simon-king-jena]

comment:84

Hey Travis,

your changes look good to me. Probably I want to have a closer look at the big patch today. But if you think the big patch is fine, then it is OK.

Apply trac11935_weak_pickling_by_construction-nt.patch trac_11935-weak_pickling_by_construction-review-ts.patch

[simon-king-jena]

comment:85

When applying the patches, sage -br gives me

----------------------------------------------------------------------
| Sage Version 5.9.rc0, Release Date: 2013-04-25                     |
| Type "notebook()" for the browser-based notebook interface.        |
| Type "help()" for help.                                            |
----------------------------------------------------------------------
**********************************************************************
*                                                                    *
* Warning: this is a prerelease version, and it may be unstable.     *
*                                                                    *
**********************************************************************
/home/simon/SAGE/prerelease/sage-5.9.rc0/local/lib/python2.7/site-packages/sage/categories/category.py:1140: UserWarning: Objects.HomCategory.ParentMethods should not have a super class
  warn("%s.%s should not have a super class"%(cls.__name__, method_provider))

**********************************************************************

Oops, Sage crashed. We do our best to make it stable, but...

A crash report was automatically generated with the following information:
  - A verbatim copy of the crash traceback.
  - A copy of your input history during this session.
  - Data on your current Sage configuration.

It was left in the file named:
        '/home/simon/.sage/ipython-0.12/Sage_crash_report.txt'
If you can email this file to the developers, the information in it will help
them in understanding and correcting the problem.

You can mail it to: sage-support at sage-support@googlegroups.com
with the subject 'Sage Crash Report'.

If you want to do it now, the following command will work (under Unix):
mail -s 'Sage Crash Report' sage-support@googlegroups.com < /home/simon/.sage/ipython-0.12/Sage_crash_report.txt

To ensure accurate tracking of this issue, please file a report about it at:
http://trac.sagemath.org/sage_trac

[simon-king-jena]

comment:86

sage -syncbuild did not help. What can I do?

[simon-king-jena]

comment:87

I think it is "needs work". Without the two patches, sage starts fine. But with the first patch, it crashes.

[simon-king-jena]

comment:88

Patchbot confirms the problem.

[tscrim]

comment:89

Hey Simon,

Hmmm... it works for me on 5.10.beta3 with both patches applied. Although I'm using the version of Nicolas' patch in the combinat queue, but there are no real differences between the two patches:

diff trac11935_weak_pickling_by_construction-nt.patch ~/Downloads/trac11935_weak_pickling_by_construction-nt.patch 
4,5c4,5
< # Node ID 3045fe70aaf8fb0c0d9d3551944317ee0764c7da
< # Parent f5ad6657cff1a065b4192e84a96fca0bc597c2c7
---
> # Node ID 0e23d0ca3d0bb47caa234d9d1890f7c603f3ddbd
> # Parent  f5ad6657cff1a065b4192e84a96fca0bc597c2c7

I don't quite understand the new patchbot logs yet, so but have you tried with both patches applied Simon (although I'd be somewhat surprised if my review patch unintentionally fixed something).

Also I didn't look at the patch functionality, just the docstrings/tests since this is what Nicolas stated as needing to be reviewed.

[simon-king-jena]

comment:90

Replying to @tscrim:

Hmmm... it works for me on 5.10.beta3 with both patches applied. Although I'm using the version of Nicolas' patch in the combinat queue, but there are no real differences between the two patches:

I suspect that there is a dependency that I did not apply. But in any case: The message Objects.HomCategory.ParentMethods should not have a super class just seems plain wrong to me. I believe that of course one should be able to allow super classes for parent methods.

[simon-king-jena]

comment:91

Nicolas, can you elaborate why parent classes should have no super class? And do you have an idea what dependencies make this patch work? Certainly these are new dependencies, because the patchbot reports the failure with sage-5.10.beta3.

[simon-king-jena]

comment:92

PS: Yes, Travis, I get the failure with both patches and with the first patch only.

[simon-king-jena]

comment:93

Aha, the user warning about super classes has been introduced in the first patch. So, Nicolas, what is the rationale? I would actually expect that some parent classes will have a super class, namely Ring or Homset and so on.

[simon-king-jena]

comment:94

PS: I think I have even posted a patch on some trac ticket in which I make Homsets use Sage's coercion model, so that their custom __call__ can be removed. But this will only work when named classes can be constructed with super classes.

[simon-king-jena]

comment:95

Replying to @simon-king-jena:

PS: I think I have even posted a patch on some trac ticket in which I make Homsets use Sage's coercion model, so that their custom __call__ can be removed.

For the record: This is #14279

[nthiery]

Changed dependencies from #9138, #11900, #11943, #12875, #12877 to #9138, #11900, #11943, #12875, #12876, #12877

[nthiery]

comment:97

Replying to @simon-king-jena:

Replying to @tscrim:

Hmmm... it works for me on 5.10.beta3 with both patches applied. Although I'm using the version of Nicolas' patch in the combinat queue, but there are no real differences between the two patches:

I suspect that there is a dependency that I did not apply.

Oops, sorry; I had tried last week to remove #12876 as dependency, and forgot to put it back into place. Fixed!

[nthiery]

comment:98

Hi Travis, Simon,

Replying to @simon-king-jena:

your changes look good to me.

+1. Thanks Travis for tidying the documentation!

Probably I want to have a closer look at the big patch today. But if you think the big patch is fine, then it is OK.

Please! I think it's a good time for both of us to look over our shoulders and see what we think of the change with a good step back.

[simon-king-jena]

comment:99

Replying to @nthiery:

Please! I think it's a good time for both of us to look over our shoulders and see what we think of the change with a good step back.

Please don't forget to tell why you insist on not allowing parent or element classes to inherit from anything useful ;-P

[nthiery]

comment:100

Hi Simon!

Thanks for putting the rational of this patch to trial!

Let me call "category class" the element/parent/... classes of categories. Here is my rationale for having them not inherit from anything (the fact that they did in a couple exceptional cases has always been in my mind a temporary hack to ease the transition to categories).

• From a OO point of view, we want to keep category classes as close possible from the model of a hierarchy of abstract classes providing generic code.

  Homset is a concrete class (in the sense that it provides data structure, ...); so having a category class inherit from it deviates from this model.

• This makes the model simpler to explain: C.element_class contains code provided by C and its upper categories. Nothing else. The hierarchy of classes for parents / elements is perfectly parallel to that of categories.

• This allows to give a clean description of the mro of a parent (or an element). First a sequence of concrete classes, related by single inheritance, that roughly speaking describe the data structure and basic methods of the object:

  E.g. CombinatorialFreeModule -> Parent -> CategoryObject -> SageObject

  Then, all the category classes that give the generic algorithm. All mutiple inheritance is concentrated in this part of the mro.

  This leaves full freedom for organizing it to avoid conflicts (like in the upcoming C3 patch) or to do further optimizations. If a category class inherit from, say, Homset, then Homset will appear in the middle of category classes in the mro, and this is more likely to create mro conflicts.

My theory is that we should always be able to avoid inheritance of category classes upon anything else; and in fact that this should actually point us to cleaner design. Now I'd be more than happy to test this against concrete situations like #14279.

The logic in Hom is such that every homset is anyway constructed as an instance of the concrete class Homset or some of its subclasses (RingHomset, ...). Could you point me to the exact problem that arised when you worked on it?

Cheers, Nicolas

[nthiery]

comment:101

Replying to @simon-king-jena:

Please don't forget to tell why you insist on not allowing parent or element classes to inherit from anything useful ;-P

I was working on it :-)

By the way: this patch is likely to conflict with the cythonization of homsets, right? How bad do you expect it would be to rebase #14214 on top of this one?

Cheers, Nicolas

[simon-king-jena]

comment:102

Replying to @nthiery:

By the way: this patch is likely to conflict with the cythonization of homsets, right? How bad do you expect it would be to rebase #14214 on top of this one?

Shouldn't be a big problem.

[nthiery]

comment:103

Replying to @simon-king-jena:

Shouldn't be a big problem.

Ok, good. Sorry for the overhead though!

[simon-king-jena]

comment:104

Earlier today, I wrote a long reply, but my laptop's battery seems to be damaged, and without a warning the computer shut down and the whole text was lost. Bad.

Replying to @nthiery:

• This makes the model simpler to explain: C.element_class contains code provided by C and its upper categories. Nothing else. The hierarchy of classes for parents / elements is perfectly parallel to that of categories.

OK. But at some point, we want to mix the abstract class with a concrete class. For example, if P is a parent, we have P.element_class, which mixes an abstract class P.category().element_class with a concrete class P.Element.

The logic in Hom is such that every homset is anyway constructed as an instance of the concrete class Homset or some of its subclasses (RingHomset, ...).

Exactly.

So, in your approach, where would this mix happen? Of course, if a homset H is initialised as an object of, say, Rings().hom_category(), and if the base class of H is a python class, then of course the class of H is changed into a sub-class of Rings().hom_category().parent_class.

But the first problem arises if the base class of H is an extension class. Then, H.__class__ can't be overridden, and thus inheritance from the abstract class is by the custom __getattr__ of Parent, which is slow compared with proper inheritance.

The second problem is in Hom(X,Y,category). First X._Hom_(Y,category) is attempted, but if this doesn't return anything useful, the abstract class C.hom_category().parent_class is instantiated.

    H = category.hom_category().parent_class(X, Y, category = category)

But an abstract class should not be instantiated! This is exactly why I thought it would be better to mix the abstract and concrete classes by providing C.hom_category().ParentMethods with bases.

And even worse things happen in sage.categories.rings.Rings.HomCategory.ParentMethods.__new__, which is a very ugly hack that should be avoided IMHO.

If you don't like bases, what do you think of the following alternative model? I try to be as close to P.element_class (P being a parent) as possible.

Let C be a category. On the one hand, it shall be provided with a concrete class C.Homset defaulting to sage.categories.homset.Homset, which could be overridden with a more concrete sub-class such as sage.rings.homset.RingHomset_generic. This corresponds to P.Element.

As we have discussed already, if S is a subcategory of C, then we would not want that S.Homset is obtained from C.Homset, unless S is a full subcategory. But fortunately, when we set Rings().Homset = RingHomset_generic, then Algebras().Homset would still default to Homset, not RingHomset_generic.

On the other hand, we get an abstract class C.hom_category().parent_class. This corresponds to P.category().element_class.

I suggest that the concrete and abstract classes be mixed in a lazy attribute C.homset_class, in the same way as the concrete and abstract classes are mixed in P.element_class.

If I am not mistaken, we discussed this in one of my visits to Orsay. If I recall correctly, the only concern has been improper inheritance: When we have a non-full subcategory S of C, then S.homset_class should not inherit from C.homset_class.

But this, I think, is something that can be solved by making S.hom_category().super_categories() return the right thing. Namely, the concrete class S.Homset is not inherited from C.Homset anyway, and the abstract class `S.hom_category().parent_class will be fine if and only if S.hom_category().super_categories() returns the right thing.

Final remark: In Hom(X,Y,category), it would still be possible for X._Hom_(Y,category) to do something special. But I believe that with the model I described it will be possible in most cases to rely on the generic (purely categorical) construction of the homset.

So, question: Do you think that this model is the way to go? It keeps abstract and concrete classes neatly apart, it will reduce the need to have custom X._Hom_(...), and we could get rid of the hack with __new__.

If you agree, then I'd say we try to finalise the patch here in the way you suggested, namely without allowing base classes. And in a second step (I think I have opened an according ticket 2 or 3 years ago) we could implement the model I sketched.

Could you point me to the exact problem that arised when you worked on it?

No. It has been a while ago, and I lost track, I am afraid.

[simon-king-jena]

comment:105

Replying to @simon-king-jena:

So, question: Do you think that this model is the way to go? It keeps abstract and concrete classes neatly apart, it will reduce the need to have custom X._Hom_(...), and we could get rid of the hack with __new__.

I think I should first try to review #12876 (which is a dependency anyway). There, you state:

• Unified the logic for selecting the class when building a Homset (e.g. Homset, RingHomset, HeckeModuleHomspace, ...). This is now systematically done through the _Hom_ hook. The logic still has a fundamental flaw, but that's for the later #10668.

So, you want that it is through the _Hom_ hook---let me guess (haven't read it yet): _Hom_ will play a similar role to Hom as _element_constructor_ plays to __call__...

[nthiery]

comment:106

Hi Simon!

Thanks for your work on this patch!

Replying to @simon-king-jena:

Earlier today, I wrote a long reply, but my laptop's battery seems to be damaged, and without a warning the computer shut down and the whole text was lost. Bad.

Ouch. I got struck by this as well, and this is very annoying. I am now protecting myself from such loss by using the cool "It's all text" extension to firefox to edit the trac fields in my favorite editor, and making sure that the temporary file used for that actually lives in my home directory and not /tmp.

Replying to @nthiery: OK. But at some point, we want to mix the abstract class with a concrete class. For example, if P is a parent, we have P.element_class, which mixes an abstract class P.category().element_class with a concrete class P.Element.

Indeed, P.element_class and C.element_class have different status. The former is a concrete class, while the later is an abstract class. I guess I should have been more specific: the hierarchy of category classes for parents (i.e. all the C.parent_class's) is perfectly parallel to that of categories (all the C's).

But the first problem arises if the base class of H is an extension class. Then, H.__class__ can't be overridden, and thus inheritance from the abstract class is by the custom __getattr__ of Parent, which is slow compared with proper inheritance.

Right, but that's no different than for other parents and element. We have lived with it so far, and as long as Cython won't allow multiple inheritance from purely abstract classes we will have to. But there is hope: Robert mentioned several times that there is no theoretical barrier and that it just needs to be implemented :-)

The second problem is in Hom(X,Y,category). First X._Hom_(Y,category) is attempted, but if this doesn't return anything useful, the abstract class C.hom_category().parent_class is instantiated.

    H = category.hom_category().parent_class(X, Y, category = category)

Not anymore! Line 306 of homset.py, with the patch applied, if no _Hom_ method is provided by the parent or the parent class, then Hom resorts to:

            H = Homset(X, Y, category = category)

which creates H as an instance of the concrete base class Homset with the appropriate category.

And even worse things happen in sage.categories.rings.Rings.HomCategory.ParentMethods.__new__, which is a very ugly hack that should be avoided IMHO.

You mean in sage.categories.schemes, right?

Yikes. I had forgotten about this old hacky workaround I had introduced at the beginning of categories. I am pretty sure we can now get rid of it using an appropriate _Hom_ method in Schemes.ParentMethods. And indeed, this ticket would be a good time to get rid of it. I'll have a look!

If you don't like bases, what do you think of the following alternative model? ...

I agree that all _Hom_ methods are basically doing the same thing: specifying what concrete homset class to do. So your approach could remove a bit of duplication. On the other hand, the _Hom_ approach is flexible and has already been used for a while in Sage. And it's already implemented :-)

As for the homset class: we had indeed discussed this in Orsay, and the plan is described in #10668. Basically, as you point out, super_categories for Hom categories should be fixed so that C.HomCategory() is a super category of S.HomCategory() only if C is a full subcategory of S. Then nothing special needs to be done for abstract homset classes in categories; they are just regular abstract parent classes. On the other hand something needs to be done for morphisms, so that a morphism in C gets code for morphisms of S. To implement this, we want to have C.morphism_class and associated MorphismMethods, with inheritance done properly.

That's actually partially implemented in the (disabled):

http://combinat.sagemath.org/patches/file/tip/category-hom_methods-nt.patch

and looked like it was going to work smoothly.

So, question: Do you think that this model is the way to go? It keeps abstract and concrete classes neatly apart, it will reduce the need to have custom X._Hom_(...), and we could get rid of the hack with __new__.

If you agree, then I'd say we try to finalise the patch here in the way you suggested, namely without allowing base classes. And in a second step (I think I have opened an according ticket 2 or 3 years ago) we could implement the model I sketched.

I'd say: as you suggest, let's finish the patch as is (hopefully with the __new__ removed). Then let's move on to the other functorial construction patch. Then cythonization of homsets / #10668 / ... Then, with the experience, if we find ourselves implementing _Hom_ methods too often, we can introduce a nicer protocol like the one you propose to only provide a class.

I guess one point of it is that I don't have a good view on whether we will have so many concrete classes for homsets (for morphism that's a different story!).

Could you point me to the exact problem that arised when you worked on it?

No. It has been a while ago, and I lost track, I am afraid.

Ok. Let me know if you stumble again on it!

Cheers, Nicolas

[simon-king-jena]

comment:107

Hi Nicolas,

Replying to @nthiery:

But the first problem arises if the base class of H is an extension class. Then, H.__class__ can't be overridden, and thus inheritance from the abstract class is by the custom __getattr__ of Parent, which is slow compared with proper inheritance.

Right, but that's no different than for other parents and element. We have lived with it so far, and as long as Cython won't allow multiple inheritance from purely abstract classes we will have to. But there is hope: Robert mentioned several times that there is no theoretical barrier and that it just needs to be implemented :-)

I don't see how multiple inheritance comes into play. What we want is to override the attribute __class__ of an instance. That's possible in Python, but impossible in Cython, and I don't think this can ever change.

So, it is not an option to use _initalise_category_ to enrich the class of a homset H, if H.__class__ is a cython class.

But what we can do, if I am not mistaken: Create a dynamic class that is constructed from one concrete base class (which can be a cythoned Homset, for example) and an abstract class (which can be C.hom_category().parent_class). And then we could create H as an instance of this dynamic class.

Not anymore! Line 306 of homset.py, with the patch applied, if no _Hom_ method is provided by the parent or the parent class, then Hom resorts to:

            H = Homset(X, Y, category = category)

which creates H as an instance of the concrete base class Homset with the appropriate category.

Really? So, how does the abstract class comes into play, then? I guess I should look at the patch...

And even worse things happen in sage.categories.rings.Rings.HomCategory.ParentMethods.__new__, which is a very ugly hack that should be avoided IMHO.

You mean in sage.categories.schemes, right?

No, in the location I stated. It is:

sage: sage.categories.rings.Rings.HomCategory.ParentMethods.__new__??
Type:       instancemethod
String Form:<unbound method HomCategory.ParentMethods.__new__>
File:       /home/simon/SAGE/prerelease/sage-5.9.rc0/local/lib/python2.7/site-packages/sage/categories/rings.py
Definition: sage.categories.rings.Rings.HomCategory.ParentMethods.__new__(cls, X, Y, category)
Source:
            def __new__(cls, X, Y, category):
                """
                    sage: Hom(QQ, QQ, category = Rings()).__class__                  # indirect doctest
                    <class 'sage.rings.homset.RingHomset_generic_with_category'>

                    sage: Hom(CyclotomicField(3), QQ, category = Rings()).__class__  # indirect doctest
                    <class 'sage.rings.number_field.morphism.CyclotomicFieldHomset_with_category'>
                """
                from sage.rings.homset import RingHomset
                return RingHomset(X, Y, category = category)

If you agree, then I'd say we try to finalise the patch here in the way you suggested, namely without allowing base classes. And in a second step (I think I have opened an according ticket 2 or 3 years ago) we could implement the model I sketched.

I'd say: as you suggest, let's finish the patch as is (hopefully with the __new__ removed). Then let's move on to the other functorial construction patch. Then cythonization of homsets / #10668 / ... Then, with the experience, if we find ourselves implementing _Hom_ methods too often, we can introduce a nicer protocol like the one you propose to only provide a class.

OK.

Best regards,

Simon

[nthiery]

comment:108

Replying to @simon-king-jena:

I don't see how multiple inheritance comes into play. What we want is to override the attribute __class__ of an instance. That's possible in Python, but impossible in Cython, and I don't think this can ever change. So, it is not an option to use _initalise_category_ to enrich the class of a homset H, if H.__class__ is a cython class.

That's to be discussed with Robert, but I am pretty sure there will be a way out if we don't think too much in the box. Anyway, let's not worry about that for now.

But what we can do, if I am not mistaken: Create a dynamic class that is constructed from one concrete base class (which can be a cythoned Homset, for example) and an abstract class (which can be C.hom_category().parent_class). And then we could create H as an instance of this dynamic class.

A quick way being to define Homset as a Python subclass of the cythonized Homset. And the _initialize_category_ will do that for you.

Really? So, how does the abstract class comes into play, then?

Homset is a parent like all the others, and inherit code from its categories (in that case a Homcategory).

I guess I should look at the patch...

Please!

No, in the location I stated. It is:

sage: sage.categories.rings.Rings.HomCategory.ParentMethods.__new__??

Oh, right, I had forgotten about this one. This is gone with the patch; which is why I did not find it :-)

And if I was able to get rid of it, I don't see why this would not be the case for schemes as well.

Cheers, Nicolas

[nthiery]

comment:109

Replying to @nthiery:

And if I was able to get rid of it, I don't see why this would not be the case for schemes as well.

I just looked, and it seems like the __new__ in Schemes.HomCategory.ParentMethods is not actually used at all anymore. Since Schemes.HomCategory contains nothing else interesting, I am currently running all long tests with Schemes.HomCategory completely stripped away. If this works, I will post a little supplementary patch for #12876 where it belongs naturally since the same thing is done there for Rings.HomCategory.

Cheers, Nicolas

[simon-king-jena]

comment:110

Since #12876 is done, I think this is next, right?

[simon-king-jena]

comment:111

If I am not mistaken, the work issue that made me change the status to "needs_work" was the fact that sage did not start at some point of the work. But I just tested: This is no problem any more.

Apply trac11935_weak_pickling_by_construction-nt.patch trac_11935-weak_pickling_by_construction-review-ts.patch

[simon-king-jena]

comment:112

But who can review it? I guess as the author, I am only entitled to tell whether I am fine with the changes made by Nicolas and Travis.

So, what else can I do? #12895, or is there something more urgent?

[simon-king-jena]

comment:113

For the record: I think Travis' patch is fine.

[nthiery]

comment:114

I already reviewed your things. So if you are happy with my changes, then it's good to go!

Yes, #12895 is next on the list!

Thanks!

[simon-king-jena]

comment:115

I think Nicolas' changes look fine, too. But let's wait for the tests to finish.

[tscrim]

Attachment: trac_11935-weak_pickling_by_construction-review-ts.patch.gz

[tscrim]

comment:116

All tests had passed. However I've uploaded a tweaked review patch which changes the doctest line continuations to the new format.

Apply: trac11935_weak_pickling_by_construction-nt.patch trac_11935-weak_pickling_by_construction-review-ts.patch

[nthiery]

comment:117

Ok. +1 on Travis's new patch.

[simon-king-jena]

comment:118

Travis' changes look good. So, a final make ptestlong, and then we'll hopefully get it over with.

[simon-king-jena]

comment:119

The patchbot finds that all tests pass. On my laptop, tests aren't finished yet, but nothing has failed so far.

The startup.modules script complains that now the inspect module is loaded. This is a bit surprising to me, because I thought that the inspect module is loaded during startup anyway (it is frequently used). But I think there is not much need to worry---and the startup-time plugin does not complain.

The doctest continuation plugin complains about the first patch, but I just verified that the second patch fixes it.

And given all the comments above, I hope I am no entitled to change the status to "positive_review"---complain if you disagree.

[nthiery]

comment:120

Yippee :-)

[nthiery]

comment:122

Hi Jeroen,

Replying to @jdemeyer:

Milestone changed from sage-5.10 to sage-5.11

Ah shoot, it would have been helpful to have a clean base including those patches for the work on the subsequent category patches.

I assume you are in the process of cutting a rc0? Do you have some approximate schedule for 5.11?

Thanks! Nicolas

[jdemeyer]

comment:123

Replying to @nthiery:

Ah shoot, it would have been helpful to have a clean base including those patches for the work on the subsequent category patches.

Well, given that these kind of patches are notorious for breaking things, I don't want to merge this in an rc. This patch is scheduled for sage-5.11.beta1 (with #12876 in beta0).

Do you have some approximate schedule for 5.11?

What do you mean?

[nthiery]

comment:124

Replying to @jdemeyer:

Replying to @nthiery:

Ah shoot, it would have been helpful to have a clean base including those patches for the work on the subsequent category patches.

Well, given that these kind of patches are notorious for breaking things, I don't want to merge this in an rc.

Fair enough :-)

This patch is scheduled for sage-5.11.beta1 (with #12876 in beta0).

Good point; we only need to have them in a beta, not necessarily for a final release.

Do you have some approximate schedule for 5.11?

What do you mean?

Roughly speaking when do you foresee 5.11.beta1 to be out?

Thanks!

[jdemeyer]

comment:125

Replying to @nthiery:

Roughly speaking when do you foresee 5.11.beta1 to be out?

I have no idea. It mainly depends on #13245 and #14699.

[jdemeyer]

Merged: sage-5.11.beta1