sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.42k stars 478 forks source link

Conjugacy classes and rational period functions for Hecke triangle groups #16976

Closed a1dd0ea6-9300-4f97-bb3c-0f25ba420caf closed 9 years ago

a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 10 years ago

The ticket adds several new features to Hecke triangle groups (resp. elements):

Some remarks:

  1. The code to determine class numbers/representatives is very slow at the moment. :-(
  2. The case n=infinity is excluded / not supported yet :(

Depends on #16936

CC: @videlec

Component: modular forms

Keywords: hecke triangle group conjugacy class rational period function

Author: Jonas Jermann

Branch/Commit: 7dcee34

Reviewer: Vincent Delecroix

Issue created by migration from https://trac.sagemath.org/ticket/16976

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from ff8c466 to e87adec

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

e87adecRename "forms" to "elements"
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from e87adec to 8169e6d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

8169e6dsupport enlisting a (maybe generating?) set of rational period functions for a given weight and discrimiant
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

24cd073documentation bugfixes
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from 8169e6d to 24cd073

a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 10 years ago

Work Issues: want: positive block exponents

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

5fba310always choose non-negative block exponents in the non-elliptic case
cff4898fix small documentation error
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from 24cd073 to cff4898

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

d14ee40support for the calculation of linking numbers
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from cff4898 to d14ee40

a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 10 years ago

Description changed:

--- 
+++ 
@@ -12,4 +12,10 @@
   for a given discriminant
 - improved documentation and minor additions/modifications

-The code to determine class numbers/representatives is very slow at the moment. :-(
+Some remarks:
+
+1. The code to determine class numbers/representatives is very slow at the moment. :-(
+2. In some (complex) cases calculations seem to fail (see continued_fraction),
+   probably because of issues / too high complexity in underlying sage objects
+   (AA, NumberField, etc)
+3. The case n=infinity is excluded / not supported yet
a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 10 years ago

Changed work issues from want: positive block exponents to none

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

c5df163drastically simplified how conjugacy types are represented: now simply the maximal representative from the class is used (using lexicographical ordering), some documentation improvements
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from d14ee40 to c5df163

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

aad579bcheck elements by default
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from c5df163 to aad579b

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from aad579b to 6701b8d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

87df16cAnnotations to Jonas's implementation.
05e551daddress review remarks
5c0f38eMerge branch 'u/jj/theta_group' into u/jj/hecke_lseries
b6adc39Merge branch 'u/jj/hecke_lseries' into u/jj/triangle_groups
6701b8dMerge branch 'u/jj/triangle_groups' into u/jj/triangle_conjugacy
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

f8b5198bugfix: always try to simplify algebraic expressions before calculations (pari_field bug for AA?), as a consequence many operations (in particular continued fractions) are much faster, improved version of root_extension_embedding
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from 6701b8d to f8b5198

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from f8b5198 to df7816d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

bf0f14aadd text to NotImplementedError
a729353Merge branch 'u/jj/hecke_lseries' into u/jj/triangle_groups
df7816dMerge branch 'u/jj/triangle_groups' into u/jj/triangle_conjugacy
a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 10 years ago

Description changed:

--- 
+++ 
@@ -10,12 +10,10 @@
 - class number and class representatives for a given discriminant
   In particular discriminants can be listed, so can all simple/reduced elements
   for a given discriminant
+- Linking numbers
 - improved documentation and minor additions/modifications

 Some remarks:

 1. The code to determine class numbers/representatives is very slow at the moment. :-(
-2. In some (complex) cases calculations seem to fail (see continued_fraction),
-   probably because of issues / too high complexity in underlying sage objects
-   (AA, NumberField, etc)
-3. The case n=infinity is excluded / not supported yet
+2. The case n=infinity is excluded / not supported yet :(
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

46fe57aMinor reviewer changes.
fce4ea9Merge tag '6.4.beta3' into u/jj/triangle_groups
94dae50Merge branch 'u/jj/triangle_groups' into u/jj/triangle_conjugacy
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from df7816d to 94dae50

videlec commented 10 years ago
comment:17

Hello Jonas,

It is always a bad choice to make a lot of modifications in one patch (especially because no reviewer will like it). From the description of your ticket, it looks like you would better have 2 or 3.

Some remarks:

Vincent

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from 94dae50 to 8f31651

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

8f31651removed some cached_methods
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Changed commit from 8f31651 to 3c7ca0f

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 10 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

3c7ca0fadd documentation and doctests to coerce_AA
a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 10 years ago
comment:20

Hi Vincent

Thanks a lot for the feedback/review!

It is always a bad choice to make a lot of modifications in one patch (especially because no reviewer will like it). From the description of your ticket, it looks like you would better have 2 or 3.

Yeah, I realize that. I'm very sorry! Most of the changes are related/interacting.

  • Why did you choose to put the code for triangle groups in sage.modular.modform_hecketriangle. As you can see elements of arithmetic groups are in sage.modular.arithgroup while the code relative to modular forms is in sage.modular.modform. People interested in Hecke groups are not necessarily interested in modular forms.

Mainly because it was originally a stub implementation. But yes that could be changed. What about doing it in a separate ticket?

  • As you might have seen, the ArithmeticSubgroupElement class inherits directly from MultiplicativeGroupElement and not from MatrixGroupElement_generic. The reason is performance as most methods of 2x2 matrices (such as det, traces, ...) are much faster using explicit formulas. Moreover, the class for arithmetic group element only have four slots for the elements a,b,c,d which is much simpler than a list of vectors for storing rows.

Matrix group operations are used at several places. I choose MatrixGroupElement_generic because I considered it simpler / less error prone than reimplementing all operations. If it makes a big difference for performance I can try to change it. What about doing it in a separate ticket?

  • Is string_rep standard ? I thought that str would have been better.

I used 'string_rep' as a helper function for _repr_ (since I have several possible representations) but 'str' would be fine by me as well (if it isn't used for other purposes).

  • You must put useful tests in the class HeckeTriangleGroupElement and move it from the __init__ method to the main documentation of the class. These examples are here to help the users. In particular, start with code that works, not with potential failure.

The way I documented my code I put the main documentation of classes resp. their usage in the readme.py file.

  • There are too much cached_method. In an ideal world there should be none. A python objects takes some amount of RAM, and each cached method stores python objects. Abusing with the cached_method implies that you can not deal with huge list of Hecke triangle group element. You should basically remove all of them unless you have a very strong argument for having it (e.g. root_extension_embedding in HeckeTriangleGroup).

I tried to remove as many cached_methods as possible without causing a major performance drop. Some of them would cause an overall 30% drop in performance (according to doctests at least) if removed. One reason for this is also that group elements are not uniquely represented, so if the basic elements aren't cached several expensive calculations on them would be unnecessarily repeated (in particular simply displaying the element would probably take much longer).

One idea might be to use UniqueRepresentation for elements as well but in this instance I am not really sure how (resp. what to put in __classcall__). Again this is a change which is not related to the ticket so it could be done in a separate ticket.

  • Why do you use the algebraic field AA instead of number fields ? It is much slower in all situations. Note that there is a (relatively slow) method for checking that a number is positive: .is_real_positive.

.is_real_positive seems to simply use .n() to numerically decide positivity. In particular it requires some embedding into e.g. RR or RIF. I choose AA as the default embedding because it is the most versatile base for further embeddings. If I "RR" or "RIF" would be used then embeddings into AA wouldn't be easily possibly anymore.

I.e. I choose generality/flexibility/versatility over performance. If performance becomes a big issue my suggestion would be to explicitely specify an embedding for those cases. E.g. For fixed points (which live in a relative number field extension) I needed this (in particular because there is no default embedding) and there is the root_extension_embedding method for that purpose.

  • What is the point of the method m.root_extension_embedding() ? You may use directly m.parent().XXX.

For elements we know the discriminant which is given as an argument to the parent function.

Jonas

videlec commented 9 years ago
comment:24

Hi,

It is standard in Sage to have documentation directly attached to the objects. If there is a need for a larger document that would serve as explanation or tutorial, then it is fine. But it is not where the main documentation should be. Moreover, such document should be listed in the "Thematic tutorials".

I am still not happy with the cached_method. If it is long to compute the decomposition into generators from the matrix, then do not make it appear in _repr_. It is a nonsense to argue that some methods must be cached because of __repr__. Either this decomposition is important for computations, in which case it must be computed at the creation (and in parallel with the matrix multiplication). Or it is not, in which case it does not matter that it is long to compute. This concerns _basic_data and _primitive_bloc_data. On the other hand, why continued_fraction is still cached?

If the information contained in _basic_data, _primitive_bloc_data and _continued_fraction are approximately equivalent, you can choose one of them for being cached. From what I understand:

line 2700

        .. TODO:

          Improve the documentation!

Who will do that?

I do have plenty of other remarks that I can partly implement myself. But I want the one above to be settled before going further.

Vincent

a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 9 years ago
comment:25

Hi Vincent,

Thanks a lot for the response. I am sorry that you are so unhappy with the code. :(

Regarding the documentation: This could be done in a separate ticket (it relates to much more than just this ticket which is already rather large). I followed the advice of Martin Raum when I started, sorry if the documentation is in the wrong spot.

Regarding the cached methods: Of course this is not about __repr__ (it only comes into play if the representation method is changed). The result of the cached calculation is rather short compared to the possibly really long computations to get it (for all examples above). More importantly the cached results are usually reused a lot. So in my opinion the overhead of having a cached method is justified.

The information in basic_data, primitive_block_data and continued_fractions are (in my opinion) not approximately equivalent. We also don't always cache all methods. Instead different cached method have different use cases (but yes there is some hiearchy between them):

(1) If the full block data is needed (not only _primitive_block_data) then the _block_data caching helps. This is a less used case I guess but still the caching helps a lot for that case. Indeed this will also use _primitive_block_data.

(2) If only _primitive_block_data is needed, we don't need to have the _block_data, _primitive_block_data is enough. For instance "hyperbolic binary quadratic forms" are in 1-1 relation to primitive hyperbolic matrices. So I guess this is the most important case. Also: Indeed _primitive_block_data relies on continued_fraction.

(3) Block decomposition has a rather specific application (basically determining the conjugacy type). We also want to be able to cache continued fraction calculations (in particular since they're probably most costly of the cached methods) without the need to also add unrelated block decomposition calculations and data. So I left the caching for continued_fraction in place. Note that continued fractions are sometimes enough and no primitive block data is needed.

(4) _basic_data is related to the decomposition of the matrix as a product of its generators S, T. it is yet another non-equivalent use case. The whole base code is built on the basic decomposition. For instance it is used to determine if a given matrix even lies in the group. It is used heavily for calculations with modular forms / upper half plane action (to determine the automorphy factor, for calculations involving the fundamental domain or representatives in it). So I decided to always call _basic_data (unless check is disabled in which case nothing is cached by default).

Regarding UniqueRepresentation: I was just asking about it as an alternative to method caching (I myself also decided against it).

I hope the summary above helped. I understand that you are unhappy with the current implementation. I am not happy about the idea of further removing cached methods (I did some test a while ago: in each case removing a cached method had significant performance impact). But if it is an absolute requirement I would remove the caching for _block_data and _primitive_block_data. As mentioned this will however have a really large negative performance effect on methods that implicitely use them. :-(

Is the overhead of an unused cache method so large? If yes, shouldn't the caching mechanism be improved instead?

Regarding the TODO on line 2700: I added it because I am not familiar enough with all facets of the topic (in particular geometry) otherwise I would have improved it. But I know it is of interest to some people. So I guess (unfortunately) nobody will improve it. Maybe the TODO remark should be removed all together since the current documentation seems acceptable.

If the ticket modifications are better "in" than "out" then my suggestion and hope would be to commit something "close" to the current form and to apply more general modifications beyond the scope of just this ticket at a later time (in a different ticket).

Regards Jonas

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from 3c7ca0f to 0f37005

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

0f37005clarify the usage of cached methods
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

9d3bb76remove a useless TODO
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from 0f37005 to 9d3bb76

videlec commented 9 years ago
comment:28

Hey Jonas,

1) Do you mind the merge with the latest beta? That will avoid me many recompilations. Otherwise, I can rebase my two commits.

2) I wrote a commit that just clean the code. Nothing big, but a little speed up. Tell me what you think.

3) Concerning the cached_method, I decided to let only two of them (continued_fraction and _primitive_block_data) after intensive timings of the doctests. See below. Please tell me if this is really bad.

Potential caching

  1. continued_fraction
  2. _block_data
  3. _primitive_block_data
  4. primitive_power
  5. _basic_data
cached methods time relative gain
none 102.0s
all 57.8 s 43%
1 75.0 s 26%
2 79.1 s 22%
3 82.4 s 17%
4 88.0 s 13%
5 98.0 s 4%
1 + 3 64.2 s 14% (rel to 1)
2 + 3 76.0 s 4% (rel to 2)
1 + 2 67.7 s 9% (rel to 1)
1 + 2 + 3 62.3 s 3% (rel to 1+3)

So 1+3 is is a 10% lost compared to 1+2+3+4+5... but I would say that removing 3 cached_method is much more!

4) I really do not understand your naming scheme. Why decompose_basic and not word_decomposition_S_T or word_S_T or something that tells you about what it does? The same is true for many methods: _primitive_block_data, _block_data, _basic_data, block_length, decompose_block.

5) For some other methods, I was not able to understand what they do... the documentation is by far too short for me. In particular reduced_elements and simple_elements.

I am waiting for your comments before going further.

Best Vincent


New commits:

433b9d1merge 6.5.beta5
f63c738trac #16976: clean code for Hecke group element
616b048trac #16976: removed 3 out of 5 @cached_method
videlec commented 9 years ago

Changed branch from u/jj/triangle_conjugacy to public/16976

videlec commented 9 years ago

Changed commit from 9d3bb76 to 616b048

a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 9 years ago
comment:29

Hey Vincent,

1) I don't mind at all.

2) Thanks a lot. I checked all the changes very closely. They look great! :-) I made some small adjustments:

3) Thanks for the test, I didn't write mine down unfortunately. "but I would say that removing 3 cached_method is much more!": Can you maybe elaborate a bit more on the drawbacks/overheads with (unused) cached_methods? How large is the overhead? The tests (in particular interactive usages) seem to only indicate positive effects from caching. I would prefer to keep the cached methods in:

4) I tried to improve the wording slightly. "Block length" is a term used in papers.

5) I added more remarks (see is_primitive and is_simple). The terms basically relate 1-1 to the corresponding terms for the associated fixed points respectively the associated binary quadratic form (at least in the hyperbolic case). The methods reduced_elements resp. simple_elements return all corresponding elements in the conjugacy class of the given element. Ideally those should be member functions for conjugacy classes. But those aren't implemented yet (and probably will never be). :(

Best Jonas

a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 9 years ago

Reviewer: Vincent Delecroix

a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 9 years ago

Changed commit from 616b048 to 9d3bb76

a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 9 years ago

Changed branch from public/16976 to u/jj/triangle_conjugacy

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Changed commit from 9d3bb76 to 67116d5

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

433b9d1merge 6.5.beta5
f63c738trac #16976: clean code for Hecke group element
616b048trac #16976: removed 3 out of 5 @cached_method
67116d5addressed the review: renamed some functions, added 2 cached_methods back, improved documentation, fixed doctests
a1dd0ea6-9300-4f97-bb3c-0f25ba420caf commented 9 years ago
comment:32

Hi again

I would like to finnish / apply this ticket. If the cached_method are the blocker, what about a compromise: Leave primitive_power as a cached_method but remove _word_S_T_data? If that's still not enough I'm also willing to remove both if it helps to finnish the review.

Best Jonas

videlec commented 9 years ago
comment:33

Hello Jonas,

No. The number of cached_method is not a blocker (let us stick to what we have right now). I want the code to be understandable by me and usable by other people.

Vincent

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 9 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

9fabb44improve documentation