Group cohomology spkg, version 2.1.2

[simon-king-jena]

Version 2.1.2 of the modular group cohomology package is now available. There is extensive documentation. The package needs the Small Groups library (available in the gap-packages spkg) and can be installed with sage -i http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.2.spkg

Most importantly: The version 2.0, that is the current "official" version of the optional package, would not work with the current version of Sage. So, there has to be an upgrade of the spkg.

Moreover, it has new features.

New Features

1. Improved code quality

• IIRC, one of the referee's complaint on earlier versions of the package was that the !init!.py file actually contains code, rather than just importing things. So, I made a new module "factory.py". Now, the !init!.py only contains documentation and import statements.

• Moreover, I split the !call! method of the cohomology ring constructor into smaller units.

• Virtually all computations with this package require creating certain data files on disk. However, the data that are shipped by the package are installed in SAGE_DATA. So, if a user has no write access in that folder, he can read from the database, but can not add further computations to it. In previous versions, it meant that the user had to create his/her own database from scratch. This is solved in version 2.1.1: Still, the user will create his/her own database, but this database can make full use of the stuff that already is in SAGE_DATA.

• Typically one has methods whose result can be cached and needs only be re-computed if the ring approximation changes; this is now being taken care of by decorators.

• I tried to reduce compiler warnings.

• Using the package, several bugs in Singular and GAP have been uncovered. The new Singular version 3-1-1 fixes some bugs, so that in some cases more efficient methods can be used. The package tests the available Singular version; it both supports the new features and is able to work around the old Singular bugs.

• We finally achieved 100% doctest coverage!!

• By oversight, the old test script failed to execute the tests of special methods (such as _add_ or _mul_. The new script now captures all tests, and complains if a class, method or function does not appear to be tested. Also, the test script does parallel tests.

• The parallel testscript uncovered a problem with temporary files associated with Sage's pexpect interfaces -- see #10004. The testscript tests if the problem reported at #10004 is fixed, and will only do parallel testing if it is fixed.

2. Extended computational capability

• The completeness criteria are further improved. We use three criteria, namely the modified Benson criterion (found by David Green and myself), the Symonds criterion and the Hilbert-Poincaré criterion (suggested by Peter Symonds and worked out by myself). Due to new methods of constructing parameters, the Symonds criterion is now often the best choice - but not always, so that it is good to have three methods to choose from.

• New functionality includes the computation of preimages of induced homomorphisms, essential ideals and depth essential ideals. Using the package, a new example of a group was found for which the square of the essential ideal does not vanish (this is only the second known example!). So far, a conjecture of Jon Carlson on depth essential ideals can be confirmed.

3. Portability

The package fully works on t2! This is a very non-trivial achievement. It involved:

• Changing the names of several functions from C-MeatAxe, since the Sun linker seems to confuse them with functions from pari. This is a problem similar to the one found at #1396.
• t2 is a big-endian machine. GAP's random generator depends on the endianness (which is a bug). Certain internal data used by the cohomology computations are obtained by randomised algorithms of GAP. The ring presentation found for the cohomology ring depends on these internal data. So, in order to get machine independent computationally well defined results, special care was needed, as discussed on sage-devel

Testing

As usual, if the environment variable SAGE_CHECK is exported and has the value yes, the test script is automatically executed and the result saved in SAGE_ROOT/install.log.

The script tests parallely. By default, the number of threads is one third of the number of available CPUs. This can be overruled by exporting the environment variable SAGE_NUMBER_THREADS.

However, problems with parallel testing in Version 2.1 revealed a problem with Sage's pexpect interfaces: All interfaces (e.g., GAP and Singular) and all parallel branches use a single temporary file for passing long commands. A solution is at #10004. The test script of the cohomology spkg will only do parallel testing if gap and singular use different temporary files.

I successfully installed and tested the package on the following machines and the following versions of Sage and Singular:

• t2.math, Sage 4.5.1, Singular 3-1-1
• bsd.math, Sage 4.5.2, Singular 3-1-0, both in 64 and 32 bit mode
• sage.math, Sage 4.5.1, Singular 3-1-0
• x86_64 GNU/Linux, Intel Core2 CPU 6700 @ 2.66GHz, Sage 4.5.2, Singular 3-1-1 -- without write permission in SAGE_DATA.

On another GNU Linux machine with AMD processors, it builds and seems to work even with Sage 4.2.1, but apparently the interface to the @parallel decorator has changed, so that I was not able to run the test script.

optional spkg: http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.2.spkg

CC: graham.ellis@nuigalway.ie david.green@uni-jena.de

Component: packages: optional

Keywords: modular group cohomology solaris t2

Author: Simon King

Reviewer: Karl-Dieter Crisman, John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/9894

[simon-king-jena]

comment:39

Speed

I will use the occasion to slightly modify the MeatAxe sources (they already are modified compared with release 2.2.4):

MeatAxe uses plain loops to initialise memory. Actually, Michael Ringe did replace memset by such loops, in Revision 2.7. However, he does not explain why he did so.

Since most compilers should be pretty well in optimising memset (or calloc), I re-introduce the usage of memset and also of calloc. With that change, the multiplication time for two 2000x2000 matrices over GF(125) drops from 15 seconds (which already beats the default Matrix_modn_dense in Sage by a loooooong way!) to less than 12 seconds on my machine.

In other words, the cohomology computations should now generally be a bit faster.

_sig_on/_sig_off

Concerning sig_on_count: It is really a nice feature that the balance of _sig_on/_sig_off is automatically tested since sage-4.7.1! I went through the sources, and indeed it seems that there can be situations where an error is raised between _sig_on/_sig_off (which must be avoided).

The other doctest failures

• The test pGroupCohomology.factory.CohomologyRingFactory.get_public_db verifies that by default the public cohomology data base is in SAGE_DATA. Apparently that is not the case for John's Sage installations, and thus I'll remove that test.

• There is one test where non-filter-regular but algebraically independent parameters are not found in the degrees that are expected according to the doc tests. That is because algebraically dependent parameters can be found with a smaller degree sum. Hence, the completion test of Peter Symonds is used, and the program does not bother to try and find the smallest possible independent parameters. So, that test has to change.

I am now preparing an update of the package and test it against sage-4.7.1.rc2. Until that's done, the ticket should be "needs work".

[simon-king-jena]

comment:40

For the record: One failure was due to a change in the pivots() method of Sage's matrices: They used to return a list, but apparently now it is a tuple. I took care of it.

The package is updated, so sage -f http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.1.spkg should now give a better result...

[simon-king-jena]

comment:41

It turns out that the documentation does not build fine. So, the reviewing should be stalled until that's fixed as well.

[simon-king-jena]

Changed work issues from fix doc tests to fix documentation

[simon-king-jena]

comment:42

I am now running the test suite, but I think it should be ready for review now.

It turns out that the trouble that I had with building the documentation can hardly be avoided:

• If one has Cython code then Python can not determine the argspec.
• Hence, sage.misc.sageinspect.sage_getargspec tries to parse the argspec by reading the source file.
• Here, the sources are in an spkg, and thus sage_getargspec can not read the sources.
• By consequence, the argument list can not be figured out.

Since I define the arguments in the doc string, it is fine though, and it can't be changed anyway.

[simon-king-jena]

Changed work issues from fix documentation to none

[simon-king-jena]

comment:43

On my machine, all tests pass. The package is updated, so, feel free to do

sage -f http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.1.spkg

now. Also I have updated the documentation

[jhpalmieri]

comment:44

On the two OpenSolaris machines I have access to (David Kirkby's machine hawk, and the skynet machine fulvia), I'm getting one doctest failure:

File "/export/home/palmieri/.sage/temp/hawk/619/dir_0/file_14.py", line 8:                         
    sage: if sys.byteorder == 'little':                                                            
        print hash(M) == Integer(7606091044269354279)  # indirect doctest                          
    else:                                                                                          
        print hash(M) == Integer(1060097699)   # indirect doctest                                  
Expected:                                                                                          
    True                                                                                           
Got:                                                                                               
    False 

Otherwise, all tests pass on various platforms.

[simon-king-jena]

comment:45

It turns out that the package does not work with the latest sage version: It fails to install, since apparently "singular" is undefined in one method, and the new version of Cython complains if stuff is not defined. I think this is going to be 2.1.2, also with some improvements in the underlying meataxe wrapper.

[simon-king-jena]

Work Issues: make it work with the latest Sage version; perhaps 2.1.2

[simon-king-jena]

comment:46

In fact, there are several reasons for this spkg to fail in the current Sage version. The first is the new Cython version, as explained above.

The second is the new Singular version: I see a many errors of the form

// ** int division with `/`: use `div` instead in line >>  sage3687[1,tmp_i]=sage3687[1,tmp_i]**(COHO5dvNew[tmp_i]/COHO5dvOld[tmp_i]); <<

raised by Singular.

The third is signal handling: Most doctest failures just complain about the sig_on/sig_off count (they are supposed to come in pairs, but apparently I sometimes forgot to insert sig_off in the correct place. Thus, it seems that the doctest framework did not count sig_on/sig_off yet, when I last tested the cohomology spkg.

[simon-king-jena]

Description changed:

--- 
+++ 
@@ -1,5 +1,8 @@
-Version 2.1.1 of the modular group cohomology package is now available. There is extensive [documentation](http://sage.math.washington.edu/home/SimonKing/Cohomology/index.html). The package needs the Small Groups library (available in the gap-packages spkg) and can be installed with `sage -i http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.1.spkg`
+Version 2.1.2 of the modular group cohomology package is now available. There is extensive [documentation](http://sage.math.washington.edu/home/SimonKing/Cohomology/index.html). The package needs the Small Groups library (available in the gap-packages spkg) and can be installed with `sage -i http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.2.spkg`

+Most importantly: The version 2.0, that is the current "official" version of the optional package, would not work with the current version of Sage. So, there has to be an upgrade of the spkg.
+
+Moreover, it has new features.

 **__New Features__**

@@ -19,7 +22,7 @@

   - We finally achieved **100% doctest coverage!! **

-  - By oversight, the old test script failed to execute the tests of special methods (such as `__add__` or `__mul__`. The new script now captures all tests, and complains if a class, method or function does not appear to be tested. Also, the test script does parallel tests.
+  - By oversight, the old test script failed to execute the tests of special methods (such as `_add_` or `_mul_`. The new script now captures all tests, and complains if a class, method or function does not appear to be tested. Also, the test script does parallel tests.

   - The parallel testscript uncovered a problem with temporary files associated with Sage's pexpect interfaces -- see #10004. The testscript tests if the problem reported at #10004 is fixed, and will only do parallel testing if it is fixed.

[simon-king-jena]

comment:47

I have created a new version 2.1.2, available as stated in the ticket description.

There were three independent problems:

• Singular upgrade: Some commands have changed, and so I had to rewrite some parts of my code
• Cython upgrade: Some code, that admittedly was buggy, might have resulted in an error at running time, and the new Cython is refusing it during compilation. So, I fixed these bugs.
• Coercion and category framework: Some recent changes in #9138 or #11900 and related stuff, due to yours truly, have created an incompatibility with the group cohomology package: I shot myself into the foot. Now, the cohomology rings are making better use of category and coercion framework.

For me, the spkg test suite works. I didn't test on t2 or related machines, though.

Anyway, ready for review again!

[simon-king-jena]

Changed work issues from make it work with the latest Sage version; perhaps 2.1.2 to none

[jhpalmieri]

comment:48

With sage-5.0.beta8-gcc, testing seems to hang on sage.math:

Successfully installed p_group_cohomology-2.1.2
Running the test suite for p_group_cohomology-2.1.2...
Testing the package pGroupCohomology...
Will use 9 parallel processes

pGroupCohomology.dickson has no doctest
pGroupCohomology.dickson.DICKSON.__cmp__ passes in 5.80 s
pGroupCohomology.dickson.DICKSON.__call__ passes in 5.79 s
pGroupCohomology.dickson.DICKSON.__init__ passes in 5.89 s
pGroupCohomology.dickson.DICKSON.V passes in 5.92 s
pGroupCohomology.dickson.DICKSON passes in 6.71 s

At this point it didn't do anything for half an hour, so I killed it.

On an OS X 10.7 box, same Sage release, I got one doctest failure:

There were doctest failures:
pGroupCohomology.cohomology.COHO:
sage -t -optional -long "/Users/palmieri/.sage/temp/jpalmieri538.math.washington.edu/74551/dir_0/file_25.py"
**********************************************************************
File "/Users/palmieri/.sage/temp/jpalmieri538.math.washington.edu/74551/dir_0/file_25.py", line 104:
    sage: H = CohomologyRing(8,3, from_scratch=True)
Expected:
        Computing from scratch
    Group data are rooted at ...
        Initialize cohomology ring of D8
        Computing data for Small Group number 3 of order 8
        > export action matrices
        Inserting SmallGroup(2,1) as a subgroup
        Inserting SmallGroup(4,2) as a subgroup
        Computing Dickson invariants in elementary abelian subgroup of rank 2
Got:
            Computing from scratch
    Group data are rooted at /Users/palmieri/.sage/temp/jpalmieri538.math.washington.edu/6162/tmp_0/
            Initialize cohomology ring of D8
            Computing data for Small Group number 3 of order 8
            > export action matrices
            Inserting SmallGroup(2,1) as a subgroup
            Inserting SmallGroup(4,2) as a subgroup
                The state of this cohomology ring is expected to be provided at /Users/palmieri/.sage/temp/jpalmieri538.math.washington.edu/6162/tmp_0/4gp2/dat/State.sobj
            Computing Dickson invariants in elementary abelian subgroup of rank 2

This one looks pretty innocuous to me...

I'm going to run doctests on OpenSolaris soon.

[jhpalmieri]

comment:49

A number of failures on OpenSolaris. Here is a log file.

[simon-king-jena]

comment:50

Replying to @jhpalmieri:

A number of failures on OpenSolaris. Here is a log file.

It seems that all tests fail, since there is a very basic problem:

ValueError: Can not create a symbolic link to the public database. Please remove /export/home/palmieri/.sage/pGroupCohomology/db/64gp158/

Why can the symbolic link not be created? Is that an openSolaris problem?

The rationale for introducing symbolic links was as follows:

• The package comes with a data base, that is installed into SAGE_DATA/pGroupCohomology. The package may be installed by root. But even in that case, it should be possible for all users to read from the database, and to extend partial computations found in the data base.
• While reading from the data base should not be a problem, extending partial computations IS a problem, if the user has no writing permission in SAGE_DATA.
• However, the user has writing permission in his/her home directory, in particular in DOT_SAGE. So, the basic idea is to put all new data there (in a sub-folder), but get existing data from SAGE_DATA.
• It would be a wast of storage space to actually copy the data from SAGE_DATA into DOT_SAGE. Therefore I only create links in DOT_SAGE that point to data in SAGE_DATA.
• Hard links are problematic, because it may happen that DOT_SAGE and SAGE_DATA belong to different partitions, but hard links have to stay in one partition. That restriction does not apply to symbolic links.

So, that is why I work with symbolic links. And when I tried on T2 (which is now not available), it did work. Do you know a reason why it fails on your i386-pc-solaris2.11?

Since the error message suggests to remove something from /export/home/palmieri/.sage/pGroupCohomology/db/, could you perhaps try to empty that folder, and repeat the tests?

[simon-king-jena]

comment:51

Replying to @jhpalmieri:

        Inserting SmallGroup(4,2) as a subgroup
            The state of this cohomology ring is expected to be provided at /Users/palmieri/.sage/temp/jpalmieri538.math.washington.edu/6162/tmp_0/4gp2/dat/State.sobj

Hm. That message sounds like data have been moved accross directories.

Anyway. I just see that I get a couple of failures on sage-5.0.beta8, built with the gcc spkg from #12369. In some cases, problems with symbolic links are mentioned, or with files that can't be opened. Not good.

The good news is that I don't need to build Sage on OpenSolaris in order to replicated the trouble with symbolic links...

[jhpalmieri]

comment:52

On OpenSolaris, I deleted the ./sage/pGroupCohomology directory and reran self-tests. I got just a few failures: see the log file. I may be misreading it, but these look like optional doctests failing.

[simon-king-jena]

comment:53

Replying to @jhpalmieri:

On OpenSolaris, I deleted the ./sage/pGroupCohomology directory and reran self-tests. I got just a few failures: see the log file. I may be misreading it, but these look like optional doctests failing.

All but one error comes from a test that requires internet connection and tests whether the package can read download data from some repository on sage.math - and now I wonder: Isn't sage.math down at the moment?

I have just tried myself, but it failed:

sage: from pGroupCohomology import CohomologyRing
sage: H=CohomologyRing.web_db('8gp3') 
---------------------------------------------------------------------------
ReadError                                 Traceback (most recent call last)
...
ReadError: file could not be opened successfully

Indeed, the problem is on the side of sage.math, and also it doesn't help to change to boxen.math. Namely, Sage is supposed to get a g-zipped tar file out of the data base, but what it receives is an html page saying that the requested page can not be found on the server. I will ask on sage-devel what happened.

One test is about the hash of matrices. The original aim was to test against a specific value, but perhaps that was a bad idea. After all, the specific value depends on whether the machine is big or little endian, and whether it is 32 or 64 bit. So, we have four possible values.

I think the natural solution is to not test against a specific value, but against the construction of the hash.

[simon-king-jena]

comment:54

For the record: William said that the data base at sage.math.washington.edu/home/lmfdb is lost.

I hope I find some backup files, either in my home directory on sage.math, or in Jena.

[simon-king-jena]

comment:55

I have updated the spkg (at the old location).

Changes: I modified the hash function of my MeatAxe matrix wrapper. Before, the hash was based on the triple given by the modulus of the finite field, the dimensions of the matrix, and the contents of the memory block describing the matrix. Now, it is only based on the memory block describing the matrix.

Rationale:

• In our applications, the field is fixed, and so it does not add useful information to the hash.
• The size of the block of memory describing the matrix gives some information about the dimensions of the matrix, such that including the exact dimensions is not so important.

Also it is faster in that way:

sage: from pGroupCohomology.mtx import MTX
sage: M = MTX(5, [[randint(0,4) for _ in range(15)] for __ in range(15)])
sage: M.set_immutable()
sage: %timeit n = hash(M)
# With the new version
625 loops, best of 3: 750 ns per loop
# With the old version
625 loops, best of 3: 1.61 µs per loop

The doc test for the __hash__ method should now be machine independent. In order to provide an indirect test, I added a method that returns a string that corresponds to the block of memory describing the matrix.

On my computer, all tests pass. Concerning the doctests marked "optional - internet": Apparently they are run by the test suite, and they pass for me as well - even though the group cohomology data base in the Sage cluster is not available anymore. Fortunately, for the single purpose of testing my package, I stored one cohomology ring at sage.math.washington.edu/home/SimonKing/Cohomology/ - and that's enough for the tests to work.

Back to "needs review", then!

[jhpalmieri]

comment:56

A few questions:

• Is it possible (and easy) to build the documentation locally? If so, it would be nice to implement that in spkg-install, using the environment variable SAGE_SPKG_INSTALL_DOCS (see #10823).

• Also, can you provide the code for producing an on-line database, in case people want to host their own? Can people then set a variable to point to a different web site?

• I think a better way to disable parallel building is export MAKE="$MAKE -j 1". Appending the -j 1 at the end should override any earlier -j flags. But this would require testing. I also think that since you call make instead of $MAKE, maybe this is irrelevant anyway? This is all pretty minor, since it works as is.

On sage.math and on an OS X Lion box (running the version of Sage from #12369), all tests passed. On OpenSolaris, I'm getting a few doctest failures. Several are due to the internet tests, and this machine seems to have a very slow internet connection, so I'm not going to worry about that. The remaining failure:

pGroupCohomology.mtx.MTX.__hash__:
sage -t -optional -long "/export/home/palmieri/.sage/temp/hawk/17706/dir_0/file_14.py"
**********************************************************************
File "/export/home/palmieri/.sage/temp/hawk/17706/dir_0/file_14.py", line 8:
    sage: if sys.byteorder == 'little':
        print hash(M) == Integer(7606091044269354279)  # indirect doctest
    else:
        print hash(M) == Integer(1060097699)   # indirect doctest
Expected:
    True
Got:
    False
**********************************************************************

Any ideas about this one?

[simon-king-jena]

comment:57

Replying to @jhpalmieri:

A few questions:

• Is it possible (and easy) to build the documentation locally? If so, it would be nice to implement that in spkg-install, using the environment variable SAGE_SPKG_INSTALL_DOCS (see #10823).

Currently, it is not possible.

For creating the docs, I am using a script that is derived from the script that builds the Sage documentation. It would, of course, be possible to include the script in the spkg

However, that would not be enough. The docs should provide the argument list of each method. In order to determine the argspec of Cython methods, it is needed to be able to read the source code (see sage.misc.sageinspect). This is of course a problem for code that is outside the Sage library. My solution: I patched sage.misc.sageinspect, such that it would find the unpacked source of the spkg locally on my computer. But other users wouldn't have the unpacked spkg lying around.

Hm. I am just thinking: If the documentation is created during installation, then one does have the unpacked spkg. Still, I wouldn't like to patch sage.misc.sageinspect for that purpose. But perhaps the documentation builder script could hook into sage.misc.sageinspect.sage_getargspec?

Anyway, it would not be a straight forward solution.

• Also, can you provide the code for producing an on-line database, in case people want to host their own?

There is no ready-made method for that purpose. What one needs to do:

• Compute the cohomology (by default, the result is stored on disk). Make sure that the computation is "from_scratch" (there is an option of that name).
• In case of a p-group, the data are for a single cohomology ring are distributed into many different files in a directory. Create a gzipped tar of that directory (for each group separately, I mean). This tar file is then put into the data base.
• In case of groups that are not of prime power order, the cohomology ring is stored in a single file, and it should be this file that is put into the data base.

Do you think the following syntax (to be implemented) would be a nice addition?

sage: H = CohomologyRing(G, to_database="/path/to/database_folder")
sage: H.make()  # would compute the ring and create an entry in the database_folder

Can people then set a variable to point to a different web site?

Yes! See the method CohomologyRing.set_web_db(...).

• I think a better way to disable parallel building is export MAKE="$MAKE -j 1". Appending the -j 1 at the end should override any earlier -j flags. But this would require testing. I also think that since you call make instead of $MAKE, maybe this is irrelevant anyway?

Do I? I thought I had changed it into $MAKE. At least, I had the intention at some point.

pGroupCohomology.mtx.MTX.hash: sage -t -optional -long "/export/home/palmieri/.sage/temp/hawk/17706/dir_0/file_14.py"

---

File "/export/home/palmieri/.sage/temp/hawk/17706/dir_0/file_14.py", line 8: sage: if sys.byteorder == 'little': print hash(M) == Integer(7606091044269354279) # indirect doctest else: print hash(M) == Integer(1060097699) # indirect doctest

That's the old version of the test. Yesterday, I posted an update of the spkg, which contains a different (machine independent) test for the hash. Could you reforce installation, making sure you get the current version?

[jhpalmieri]

comment:58

Replying to @simon-king-jena:

Replying to @jhpalmieri:

A few questions:

• Is it possible (and easy) to build the documentation locally? If so, it would be nice to implement that in spkg-install, using the environment variable SAGE_SPKG_INSTALL_DOCS (see #10823).

Currently, it is not possible.

Perhaps for a future version you might consider it, depending on how much work it would be. A simple short-term solution would be to just include html docs in the spkg; they shouldn't add too much to the size. (Maybe a top-level directory docs, not under revision control.)

• Also, can you provide the code for producing an on-line database, in case people want to host their own?

Do you think the following syntax (to be implemented) would be a nice addition?

sage: H = CohomologyRing(G, to_database="/path/to/database_folder")
sage: H.make()  # would compute the ring and create an entry in the database_folder

Yes, that sounds good. Maybe also a list somewhere of which specific calculations would make a good database.

• I think a better way to disable parallel building is export MAKE="$MAKE -j 1". Appending the -j 1 at the end should override any earlier -j flags. But this would require testing. I also think that since you call make instead of $MAKE, maybe this is irrelevant anyway?

Do I? I thought I had changed it into $MAKE. At least, I had the intention at some point.

The spkg-install file says, for example

MAKE=make; export MAKE;

cd src

make

pGroupCohomology.mtx.MTX.hash: sage -t -optional -long "/export/home/palmieri/.sage/temp/hawk/17706/dir_0/file_14.py"

---

File "/export/home/palmieri/.sage/temp/hawk/17706/dir_0/file_14.py", line 8: sage: if sys.byteorder == 'little': print hash(M) == Integer(7606091044269354279) # indirect doctest else: print hash(M) == Integer(1060097699) # indirect doctest

That's the old version of the test. Yesterday, I posted an update of the spkg, which contains a different (machine independent) test for the hash. Could you reforce installation, making sure you get the current version?

You're right, I don't know how that happened. Anyway, all tests passed this time, including the internet ones.

I'm going to browse the source a bit more, but it all looks very good right now.

[simon-king-jena]

comment:59

Replying to @jhpalmieri:

Perhaps for a future version you might consider it, depending on how much work it would be. A simple short-term solution would be to just include html docs in the spkg; they shouldn't add too much to the size. (Maybe a top-level directory docs, not under revision control.)

It is 3.4MB.

Yes, that sounds good. Maybe also a list somewhere of which specific calculations would make a good database.

Or perhaps another syntax would be better. My previous suggestion was: "Declare during creation of a specific cohomology ring that it should go into a data base." But the computation of one ring typically involves the computation of rings for several subgroups (elementary abelian, Sylow, ...), and it would be reasonable to have them ALL in the data base.

Therefore, I think it would be better to have a global switch, for example

sage: CohomologyRing.create_database("/path/to/data_base")

with the effect that all subsequent cohomology computations will work as they usually do, but would additionally write the tar files into the given data base folder.

The spkg-install file says, for example

MAKE=make; export MAKE;

Oops. I will try what happens with parallel make - after all, the comment "Building in parallel is bad" is probably there for a reason (but I can't recall).

I'm going to browse the source a bit more, but it all looks very good right now.

OK. Would it be OK to postpone the doc and data base additon, or would you strongly prefer to have it in 2.1.2 already?

[simon-king-jena]

comment:60

One question: If $MAKE is make -jN, how can one find the number N?

[simon-king-jena]

comment:61

I have updated th spkg on the old location (hence, if you download it again, make sure that you have the correct version, because sometimes wget is cached!).

Changes:

• If SAGE_SPKG_INSTALL_DOCS=yes then the documentation is locally created, and copyied into SAGE_ROOT/local/share/doc/p_group_cohomology/html.
• I tried to allow parallel make, but MeatAxe would fail. I do not have the energy to fix it right now. At least, I am now reverting the old setting of MAKE as soon as MeatAxe is built, but apparently for installing the Cython modules it doesn't help.

Ready for review, anyway.

[simon-king-jena]

comment:62

PS: Concerning docs, you will find that certain files such as builder.py, sageinspect.py and sage_autodoc.py are modified versions of the corresponding files from Sage. They are not doctested here. The difference to the files from Sage is that the modified versions are able to find the source files of my cohomology spkg.

[jhpalmieri]

Changed reviewer from Karl-Dieter Crisman to Karl-Dieter Crisman, John Palmieri

[jhpalmieri]

comment:63

This looks great! Positive review. I appreciate having the documentation locally, too.

At least one thing which ought to be easy to implement in a future version: additive_order for elements: a.additive_order() should be either 1 (if a==0) or p (the characteristic of the ground field. If you ever construct cohomology rings over non-fields, you could leave it unimplemented in that case.

I found this a little strange:

sage: H0 = CohomologyRing(8,3)
sage: (H0.2 * H0.3).is_zero()
True
sage: (H0.2 * H0.3) == 0
False

Maybe comparisons or __eq__ need to be implemented, too.

[simon-king-jena]

comment:64

Replying to @jhpalmieri:

This looks great! Positive review.

Thank you very much! Finally, the old version (broken with recent Sage) can be replaced!

sage: (H0.2 H0.3).is_zero() True sage: (H0.2 H0.3) == 0 False Maybe comparisons or __eq__ need to be implemented, too.

I think that the answers are consistent, for the following reasons:

• H0.2*H0.3 is zero in the cohomology group H²(D8). As elements of that group, the product is zero, hence "is_zero()" returns True.
• However, even though the product vanishes and thus represents a relation of the cohomology ring, it still is a cocycle of degree 2. Thus, it is not equal to H0.zero_element(), which is a cocycle of degree 2. Hence, the comparison with the zero of the cohomology ring returns False.

[simon-king-jena]

comment:65

Dear John,

I am sorry for being late, but I have one question: Would you allow one last-minute change?

Namely, with the package you just reviewed, the computation of the cohomology ring of the Sylow 2-subgroup of some double cover of Suzuki group 8 (this is group number 836 of order 128), the completion test à la Peter Symonds is very time consuming - the problem was in the computation of a small subset of generators over which the ring is finite-dimensional. I found a way to speed it up. Also, I'd like to add protocol output to the method that computes it (which also means I will have to change the expected protocol output in some doc tests).

I think these changes are minor, but helpful. If you say that you would try to install it and run the test suite again, then I would update the spkg a bit later. I would also accept to postpone that change for a later spkg version.

What do you prefer?

[jhpalmieri]

comment:66

Hi Simon,

I can run the test suite in the next day or two, once you update the spkg. So it's fine with me if you want to update it. If you do, could you also post a diff here, so I can see the changes? (Just hg export tip > ... and post the resulting patch file.)

[simon-king-jena]

Diff of the latest changes in the spkg. For reviewing only

[simon-king-jena]

comment:67

Attachment: cohomology-pkg.diff.gz

Replying to @jhpalmieri:

I can run the test suite in the next day or two, once you update the spkg. So it's fine with me if you want to update it. If you do, could you also post a diff here, so I can see the changes?

I have updated my spkg (old location) and have attache a diff with respect to the version that you have reviewed earlier.

Here are the changes:

• The useSlimgb resp. useStd options had been used only in one location: There were several places where the default 'groebner' method of Singular would be used, regardless of the options. That has changed in the last revision.
• The protocol output of some methods (e.g., dependent_parameters()) are now providing protocol output before a potentially time-consuming Gröbner basis computation is started. Background: When I computed the cohomology of SmallGroup(128,836) with the previous revision, I thought that it hangs in the computation of the Gröbner basis of the relation ideal - but in fact the problem was in dependent_parameters(), which has not been clear from the protocoll output.
• In Symonds' criterion, parameters in degree one won't contribute to the degree in which the criterion detects completeness. Hence, adding all degree one generators to a given list of parameters will not change the degree bound -- but it may make the computation easier, because killing degree one generators greatly reduces the difficulty of Gröbner basis computations. Thus my changes in dependent_parameters

[jhpalmieri]

comment:69

Okay, still looks good.

[jdemeyer]

Description changed:

--- 
+++ 
@@ -55,3 +55,4 @@

 On another GNU Linux machine with AMD processors, it builds and seems to work even with Sage 4.2.1, but apparently the interface to the `@`parallel decorator has changed, so that I was not able to run the test script.

+**optional spkg**: [http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.2.spkg](http://sage.math.washington.edu/home/SimonKing/Cohomology/p_group_cohomology-2.1.2.spkg)

[haraldschilly]

comment:71

spkg updated on the server + mirros.