search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.43k stars 478 forks source link

create a class for basic function_field arithmetic for Sage #9054

Closed williamstein closed 12 years ago

williamstein commented 14 years ago

One of the first things we learned at Sage Days 21: Function Fields, is that it is not even possible to really define or even do arithmetic in function fields at all in Sage! It's amazing that this most basic arithmetic still isn't supported, but it isn't (maybe it used to be via generic machinery, but got broken...?). The point of this ticket is to create classes for standard function field structures, along with support for arithmetic. This should be organized in a way similar to number fields.

For this code, the main point is to establish an API that works solidly. It will be insanely slow. A subsequent patch will make things fast.

See also: #9069, #9051, #9094, #9095.

Note that the dependancy on #9138 is only because of a really minor change in the doctests. This ticket already has a positive review so I suspect this will get merged first. If that ticket eventually gets rejected it will be trivial to rebase the patch withouth that ticket.

Apply attachment: 9054_function_fields.patch to the Sage library.

Dependencies: sage-5.0.beta1

CC: @burcin @sagetrac-khwilson @koffie @mstreng @novoselt @pjbruin @mminzlaff @saraedum

Component: algebra

Author: William Stein, Robert Bradshaw, Maarten Derickx, Moritz Minzlaff, Julian Rueth

Reviewer: Maarten Derickx, Julian Rueth

Merged: sage-5.0.beta2

Issue created by migration from https://trac.sagemath.org/ticket/9054

burcin commented 14 years ago
comment:1

Attachment: trac_9054-part1.patch.gz

williamstein commented 14 years ago

Attachment: trac_9054-part2.patch.gz

williamstein commented 14 years ago

Attachment: trac_9054-part3.patch.gz

f43d6059-7290-4fa2-86e9-b6e3514ab403 commented 14 years ago
comment:2

There seems to be an issue with returning the base ring of a RationalFunctionField. Neither base() nor base_ring() return the correct ring:

sage: K.<t> = FunctionField(QQ); K
Rational function field in t over Rational Field
sage: R1 = K.base(); R1
Rational function field in t over Rational Field
sage: R2 = K.base_ring(); R2
Rational function field in t over Rational Field
sage: R3.<s> = QQ[]; K3 = Frac(R3); K3
Fraction Field of Univariate Polynomial Ring in s over Rational Field
sage: R3
Univariate Polynomial Ring in s over Rational Field
sage: K3.base() == R3
True
robertwb commented 14 years ago

Attachment: trac_9054-part4.patch.gz

Attachment: trac_9054-part5.patch.gz

robertwb commented 14 years ago
comment:3

Looks like you forgot to add the file function_field_order, so I wasn't able to doctest on top of your latest patch (let alone debug it). See also #9051 for added speed in the positive characteristic case.

williamstein commented 14 years ago

Attachment: trac_9054-part6.patch.gz

Attachment: trac_9054-part7.patch.gz

polynomial factorization!

jbalakrishnan commented 14 years ago
comment:4

FunctionField constructor clips names

sage: F = FunctionField(GF(7), 'bit')
sage: F.gen()
b
williamstein commented 14 years ago

ideals and orders!

williamstein commented 14 years ago

Attachment: trac_9054-part8.patch.gz

inverses of fractional ideals

williamstein commented 14 years ago
comment:5

Attachment: trac_9054-part9.patch.gz

Replying to @sagetrac-salmanhb:

There seems to be an issue with returning the base ring of a RationalFunctionField. Neither base() nor base_ring() return the correct ring:

sage: K.<t> = FunctionField(QQ); K
Rational function field in t over Rational Field
sage: R1 = K.base(); R1
Rational function field in t over Rational Field
sage: R2 = K.base_ring(); R2
Rational function field in t over Rational Field
sage: R3.<s> = QQ[]; K3 = Frac(R3); K3
Fraction Field of Univariate Polynomial Ring in s over Rational Field
sage: R3
Univariate Polynomial Ring in s over Rational Field
sage: K3.base() == R3
True

The above is correct. To get what you want, use the constant_field() method. 

sage: K.<t> = FunctionField(QQ);
sage: K.constant_field()
Rational Field
williamstein commented 14 years ago

morphisms of function fields

williamstein commented 14 years ago

Attachment: trac_9054-part10.patch.gz

robertwb commented 14 years ago

Attachment: trac_9054-part11.patch.gz

Attachment: trac_9054-part12.patch.gz

Various methods needed for #9095 (doctesets depend on #9094)

b747cca1-2282-44a5-a425-137db06022f7 commented 14 years ago
comment:6

Should be some automatic way to do the following:

K.<T> = FunctionField(GF(17))
P = T-5
f = P^5
R = K._ring
R(f.element()).valuation(R(p.element()))
williamstein commented 14 years ago

Attachment: trac_9054-part1-12.patch.gz

flattened patch that incorporates all of patches 1-12 above into a single patch.

williamstein commented 14 years ago
comment:8

Here is a link to the result of doctesting sage-4.4.4 + patches 1-12:

http://sage.math.washington.edu/home/wstein/patches/9054-part1-12.doctest.txt

The failed tests:

The following tests failed:

        sage -t  devel/sage-main/sage/matrix/matrix2.pyx # 1 doctests failed
        sage -t  devel/sage-main/sage/plot/matrix_plot.py # 0 doctests failed
        sage -t  devel/sage-main/sage/modular/abvar/morphism.py # 1 doctests failed
        sage -t  devel/sage-main/sage/modular/abvar/finite_subgroup.py # 1 doctests failed
        sage -t  devel/sage-main/sage/tests/startup.py # 1 doctests failed
        sage -t  devel/sage-main/sage/modular/modform/hecke_operator_on_qexp.py # 1 doctests failed
        sage -t  devel/sage-main/sage/categories/function_fields.py # 5 doctests failed
        sage -t  devel/sage-main/sage/rings/function_field/function_field_element.pyx # 14 doctests failed
williamstein commented 14 years ago

Description changed:

--- 
+++ 
@@ -1,3 +1,5 @@
 One of the first things we learned at Sage Days 21: Function Fields, is that it is not even possible to really define or even do arithmetic in function fields *at all* in Sage!  It's amazing that this most basic arithmetic still isn't supported, but it isn't (maybe it used to be via generic machinery, but got broken...?).  The point of this ticket is to create classes for standard function field structures, along with support for arithmetic.   This should be organized in a way similar to number fields. 

 For this code, the main point is to establish an API that works solidly.  It will be insanely slow.  A subsequent patch will make things fast. 
+
+See also: #9069, #9094, #9095.
williamstein commented 14 years ago
comment:13

NOTE: #9094 implements sqrt for polynomials, which is relevant to trac_9054-doctest.patch

koffie commented 14 years ago
comment:14

I guess the doctest patch isn't really usefull addition to sage (although making it was a usefull learning experience for Peter Bruin and me since it was our first patch). The patch fixes some bugs which are also fixed in other patches in trac. Some are indeed fixed by #9094 (although i think this can be done faster and more elegant) and another one related calculating the valuation in fraction fields is fixed by 9051-FpT_4.patch in #9051.

Since I'm quite new to developing and using trac and hg etc. I would like to know what is the best thing to do now. And especially how to deal with the relating patches wich also contain fixes for stuff happening here.

williamstein commented 14 years ago

Description changed:

--- 
+++ 
@@ -2,4 +2,4 @@

 For this code, the main point is to establish an API that works solidly.  It will be insanely slow.  A subsequent patch will make things fast. 

-See also: #9069, #9094, #9095.
+See also: #9069, #9051, #9094, #9095.
koffie commented 14 years ago
comment:16

Added an attachment that fixes all but three doctest failures. The remaining failures are:


sage -t  "devel/sage-mderickx/sage/modular/abvar/morphism.py" # 1 failure

sage -t  "devel/sage-mderickx/sage/modular/abvar/finite_subgroup.py" # 1 failure

sage -t  "devel/sage-main/sage/modular/modform/hecke_operator_on_qexp.py" # 1 failure

}}}They are all related since their error messages all end in:{{{      File "/Applications/sage/local/lib/python/site-packages/sage/modules/free_module.py", line 4700, in _echelonized_basis        d = self._denominator(basis)      File "/Applications/sage/local/lib/python/site-packages/sage/modules/free_module.py", line 4810, in _denominator        d = d.lcm(x.denominator())    !AttributeError: 'int' object has no attribute 'lcm'}}}It would be nice if someone who has a better understanding of sage to fix this final bug, since then we would have no doctests failing anymore for this patch.
koffie commented 14 years ago
comment:17

Oeps, wrong fromatting. Now a bit more readable:

sage -t  "devel/sage-mderickx/sage/modular/abvar/morphism.py" # 1 failure

sage -t  "devel/sage-mderickx/sage/modular/abvar/finite_subgroup.py" # 1 failure

sage -t  "devel/sage-main/sage/modular/modform/hecke_operator_on_qexp.py" # 1 failure

They are all related since their error messages all end in:


      File "/Applications/sage/local/lib/python/site-packages/sage/modules/free_module.py", line 4700, in _echelonized_basis
        d = self._denominator(basis)
      File "/Applications/sage/local/lib/python/site-packages/sage/modules/free_module.py", line 4810, in _denominator
        d = d.lcm(x.denominator())
    AttributeError: 'int' object has no attribute 'lcm'
koffie commented 14 years ago
comment:20

Has there been any work on this since sage days > 23? Even if the work is only partially finnished it would be good to know to avoid double work.

koffie commented 14 years ago

Attachment: trac_9054-doctest.patch.gz

Aplies to sage 4.4.4 after 1-12 patch and it also needs the #9054 patch trac_9094-sqrt-mderickx.patch

williamstein commented 14 years ago
comment:21

Replying to @koffie:

Has there been any work on this since sage days > 23? Even if the work is only partially finished it would be good to know to avoid double work.

There has been no further work. When I do work further on this, I will post a patch. I always post patches of everything I do as I go, as soon as I'm done with a session of work.

williamstein commented 14 years ago
comment:22

I am moving this entirely out of Sage and into the psage library. See

http://code.google.com/p/purplesage/issues/detail?id=3

williamstein commented 14 years ago
comment:23

I'm closing this since I'm no longer interested in getting it included in the main sage distribution. It is now in psage as mentioned above.

robertwb commented 14 years ago
comment:25

I think eventually this should be in the main sage distribution, even if no one's actively working on it right now.

188546cd-a485-46fd-ae74-488a1552fab0 commented 13 years ago
comment:26

There should be no doctest failures left. Comments, remarks, and reviews are welcome. :)

Apply trac_9054-all-parts.patch

Depends on #9094, #11034

188546cd-a485-46fd-ae74-488a1552fab0 commented 13 years ago

Description changed:

--- 
+++ 
@@ -3,3 +3,9 @@
 For this code, the main point is to establish an API that works solidly.  It will be insanely slow.  A subsequent patch will make things fast. 

 See also: #9069, #9051, #9094, #9095.
+
+---
+
+Apply trac_9054-all-parts.patch
+
+Depends on #9094, #11034
188546cd-a485-46fd-ae74-488a1552fab0 commented 13 years ago

Author: William Stein, Robert Bradshaw, Maarten Derickx, Moritz Minzlaff

saraedum commented 13 years ago
comment:29

The doctests of function_field.py contain the following lines:

sage: R.<x> = FunctionField(QQ); S.<y> = R[]
sage: R2.<t> = FunctionField(QQ); S2.<w> = R2[]
sage: L2.<w> = R.extension((4*w)^2 - (t+1)^3 - 1)

I think it is confusing that it does not make a difference whether you write R.extension or R2.extension in this example. I'm new to sage so maybe I'm misunderstanding something here.

saraedum commented 13 years ago

Attachment: trac_9054_polynomial_base_field.patch.gz

polynomial used for a field extension must be defined over the base field

saraedum commented 13 years ago
comment:30

There are some problems with the zero of a function field:

sage: K.<x> = FunctionField(QQ); R.<y> = K[]; L.<y> = K.extension(y^2+x);
sage: coerce(L,L.polynomial())==0
False
sage: y/0
0
saraedum commented 13 years ago

fixes the problems regarding zero.

saraedum commented 13 years ago
comment:31

Attachment: trac_9054_zero.patch.gz

Entering the following at the sage prompt produces a TypeError: Unable to coerce -u^2 (...) to Rational.

K.<x> = FunctionField(QQ); R.<y> = K[]
L.<y> = K.extension(y^2 - x)
M.<u> = FunctionField(QQ); R.<v> = M[]
N.<v> = M.extension(v-u^2)
L.hom([u,v])

This is due to the fact that hom() determines the codomain by looking only at the first element of [u,v].

saraedum commented 13 years ago

Attachment: trac_9054_codomain.patch.gz

set the correct codomain for function fields

saraedum commented 13 years ago

Attachment: trac_9054_doctest-2.patch.gz

fixes hash doctests for 32bit and a random doctest

saraedum commented 13 years ago
comment:32

Is there a reason why a FunctionFieldMorphism is a Map and not a RingHomomorphism?

saraedum commented 13 years ago

Attachment: trac_9054_function_fields_sd32.patch.gz

Minimal support for functions field. Does not include all of the above patches.

koffie commented 13 years ago
comment:33

I'm now busy with very troughly checking the entire patch wich at least with some changed free module stuff passes all doctests. There will be a big doctest patch comming up which includes tests I've thought up to also test some more none trivial examples.

There is are at least two big issues which I run in to today. They all occured in the same terminal session.

sage: K.<x> = FunctionField(QQ)
sage: R.<y> = K[]
sage: L.<w> = K.extension(y^5 - (x^3 + 2*x*y + 1/x));
sage: w.is_integral()
False
sage: L.order(w)  #should raise a value error since orders can only be generated by integral elements
Order in Function field in w defined by y^5 - 2*x*y + (-x^4 - 1)/x
sage: L.order(w).gens()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)

/Users/maarten/Downloads/sage-4.7.2.alpha2/devel/sage-main/<ipython console> in <module>()

/Users/maarten/Downloads/sage-4.7.2.alpha2/local/lib/python2.6/site-packages/sage/structure/parent_gens.so in sage.structure.parent_gens.ParentWithGens.gens (sage/structure/parent_gens.c:2741)()

/Users/maarten/Downloads/sage-4.7.2.alpha2/local/lib/python2.6/site-packages/sage/structure/parent_gens.so in sage.structure.parent_gens.ParentWithGens.ngens (sage/structure/parent_gens.c:2548)()

/Users/maarten/Downloads/sage-4.7.2.alpha2/local/lib/python2.6/site-packages/sage/structure/parent_gens.so in sage.structure.parent_gens.check_old_coerce (sage/structure/parent_gens.c:1228)()

RuntimeError: Order in Function field in w defined by y^5 - 2*x*y + (-x^4 - 1)/x still using old coercion framework
koffie commented 13 years ago
comment:34

the review patch is not entirely ready, but julian wanted to have a look so I uploaded it

simon-king-jena commented 13 years ago
comment:35

At sage-devel, Maarten mentioned that pickling does not seem to work for the code posted here, which seems to be due to some attributes typically involved in coercion.

Looking at attachment: trac_9054-all-parts.patch, I see that the base class for function fields is derived from sage.rings.ring.Field, but Field.__init__ is not called.

The rings in vanilla Sage do not pay enough attention to coercion and categories. #9944 and (in particular) #9138 aim at improving the situation. In particular, with #9138 it should now possible to avoid any direct call to ParentWithGens.__init__; calling Field.__init__ should just work (tm). Can you try?

simon-king-jena commented 13 years ago
comment:36

PS: After attachment: trac_9054-all-parts.patch was created, several other patches were posted. Can you please clearly state (in the ticket description and, for the patchbot, also in a comment) which patches are supposed to be applied? It is difficult to work on the pickling problem (or reviewing) if it is not clear what code exactly we are talking about.

simon-king-jena commented 13 years ago
comment:37

Here are some comments on attachment: trac_9054-all-parts.patch:

simon-king-jena commented 13 years ago
comment:38

Replying to @simon-king-jena:

  • I already mentioned,

... namely on sage-devel,

that Field.__init__(...) should be called. It could be that this only works when #9138 is used. Just calling ParentWithGens.__init__ may be insufficient.

williamstein commented 13 years ago
comment:39

Replying to @simon-king-jena:

Here are some comments on attachment: trac_9054-all-parts.patch:

  • Please remove the __contains__ method from the category FunctionFields. Containment in categories should rely on the default implementation, unless there is a compelling reason to do otherwise.

    Even worse, your containment test is ultimately based on testing class inheritance (namely in the function is_FunctionField). That totally undermines the category framework. It must be possible for an object to be a function field even without inheriting from sage.rings.function_field.function_field.FunctionField.

    The default implementation of F in FunctionFields() relies on the category of F: The containment test returns True if and only if F.category() is a sub-category of FunctionFields(). That should be much better, from a mathematical point of view, than testing class inheritance!

Technically this is true. But this category framework instead of inheritance -- really two very different approaches to design -- leads directly to slow code in some cases in practice, which is really annoying, IMHO. For example, see #11657, where one of the root causes of slowness was code in is_Ring that was added to support this category approach, and which slowed everything down. Fortunately for me I have psage where I can write streamlined code without having to be weighed down, and for generic Sage working well and being extensible is more important, so of course I agree with you in this case.

  • You should add a test of the form TestSuite(F).run(), where F is a function field. The test suite is formed by some generic tests defined in the category framework and includes many sanity tests (such as pickling for the field and its elements, associativity, commtativity, ...). If you can think of specific tests for function fields, then you should add methods named _test_... as parent or element methods of sage.categories.function_fields.FunctionFields. Such methods will be automatically called when running TestSuite(F).run().

  • You should also add a test of the form loads(dumps(F)) is F, in order to test uniqueness of parent structures; if I recall correctly, the test suite from the category would only test loads(dumps(F))==F.

This is also testing that pickling works at all. This code is used by the pickle jar to create pickles for testing later.

  • It should not be needed to have a function is_FunctionField (that just tests class inheritance) - F in FunctionFields() is a better test, IMHO. If you do want to preserve is_FunctionField then please do not simply put it in the global name space. At least, it should be deprecated, similar to is_Ring being deprecated. There is a function decorator to do so.

is_Ring is only deprecated when used from the top level (i.e., the Sage prompt). However, there is still a is_Ring function, which can be used in library code, and is not deprecated for this purpose. And the is_Ring function does test for category stuff.

  • In the doc test for the _element_constructor_ method, you explicitly call the method. I think i

t should better be an indirect test (after all, the documentation is supposed to show how the user is supposed to work with stuff). Hence, not L._element_constructor_(L.polynomial_ring().gen()) but L(L.polynomial_ring().gen()) #indirect doctest.

I disagree. I view "#indirect test" for situations where you can't think of a clean way of directly calling the function. If there is such a way, use it! That way, at least you know for sure it is really being tested. Suggesting to get rid of that makes no sense to me. What if L(L.polynomial_ring().gen()) doesn't call _element_constructor_ at all? Also, one can also just have two tests -- one that is indirect and one that isn't.

  • I already mentioned, since FunctionField is derived from sage.rings.ring.Field, that Field.__init__(...) should be called. It could be that this only works when #9138 is used. Just calling ParentWithGens.__init__ may be insufficient.

  • There are several methods, such as polynomial_ring or vector_space, that use a hand-made cache. Please use the @cached_method decorator instead! That has several reasons.

    1. It is more easy. You don't need to manually update attributes.
    2. With #11115, the @cached_method decorator is rewritten in Cython and provides a faster cache than anything you could possibly create with Python.

+1. Note that when the very first version of the function field code was written (by me) @cached_method was disturbingly slow. I really, really appreciate the fast Cython rewrite.

  • Is there a reason why you have a method base_field that simply returns the function field itself? From the behaviour of the base_ring method of polynomial rings, I would rather expect that FunctionField(QQ,['t']).base_field() returns the rational field.

No. The base field of a function field is a rational function field in 1 variable. The base field of that rational function field is then a field such as QQ. Most function fields aren't rational, e.g., they are finite extensions K/QQ(t), or even relative extensions L/K. In the first case, the base field is QQ(t) and in the second it is K.
If Simon was confused by this, it should be documented better.

simon-king-jena commented 13 years ago
comment:40

Replying to @williamstein:

Replying to @simon-king-jena:

Here are some comments on attachment: trac_9054-all-parts.patch:

  • Please remove the __contains__ method from the category FunctionFields. Containment in categories should rely on the default implementation, unless there is a compelling reason to do otherwise.

...

Technically this is true. But this category framework instead of inheritance -- really two very different approaches to design -- leads directly to slow code in some cases in practice, which is really annoying, IMHO.

A while ago, I had worked on a ticket #10667 about category containment. One purpose was to get a speedup. The trick was (again) to use Cython. For some reason, the work on that ticket has stalled. Perhaps it would be worth while to resume it.

Generally, I think it is better to improve the category framework, rather than to work around it.

For example, see #11657, where one of the root causes of slowness was code in is_Ring that was added to support this category approach, and which slowed everything down.

Then why is the existing is_Ring not rewritten along the lines of what you do in #11657?

is_Ring is only deprecated when used from the top level (i.e., the Sage prompt).

Yes, this is what I meant. I did not mean "deprecated" in the sense of "will soon be removed", but in the sense of "please don't try this at home".

And the is_Ring function does test for category stuff.

Actually I have not been aware that category stuff is tested in is_Ring. I was thinking about various other is_... methods that really do nothing more than isinstance.

  • Is there a reason why you have a method base_field that simply returns the function field itself? From the behaviour of the base_ring method of polynomial rings, I would rather expect that FunctionField(QQ,['t']).base_field() returns the rational field.

No. The base field of a function field is a rational function field in 1 variable.

Ouch, so I was mistaken.

The base field of that rational function field is then a field such as QQ. Most function fields aren't rational, e.g., they are finite extensions K/QQ(t), or even relative extensions L/K. In the first case, the base field is QQ(t) and in the second it is K.
If Simon was confused by this, it should be documented better.

Not needed. What I stated was based on reading the patch "diagonally". I only noticed one of the two base_field methods.

williamstein commented 13 years ago
comment:41

Replying to @simon-king-jena:

A while ago, I had worked on a ticket #10667 about category containment. One purpose was to get a speedup. The trick was (again) to use Cython. For some reason, the work on that ticket has stalled. Perhaps it would be worth while to resume it.

+1

Generally, I think it is better to improve the category framework, rather than to work around it.

For example, see #11657, where one of the root causes of slowness was code in is_Ring that was added to support this category approach, and which slowed everything down.

Then why is the existing is_Ring not rewritten along the lines of what you do in #11657?

What I did there slows down is_Ring testing if the object in question does not derive from Ring.

is_Ring is only deprecated when used from the top level (i.e., the Sage prompt).

Yes, this is what I meant. I did not mean "deprecated" in the sense of "will soon be removed", but in the sense of "please don't try this at home".

If you are developing on the Sage library, I think it is OK to use.

And the is_Ring function does test for category stuff.

Actually I have not been aware that category stuff is tested in is_Ring. I was thinking about various other is_... methods that really do nothing more than isinstance.

Yes, take a look at the code. I too was surprised by this!

-- William