Closed dimpase closed 5 years ago
Branch pushed to git repo; I updated commit sha1. New commits:
67ca321 | fixed gap_packages, also package compiling/installing |
Branch pushed to git repo; I updated commit sha1. New commits:
a7ef9e0 | backport guava patch to allow no gcc |
It turns out that most of the changes so far here should go to #22626. Once this is done I'll update the branch accordingly.
Is there any reason the IO package isn't installed here? It seems to be mentioned quite often, but we don't provide a means to install it.
Replying to @embray:
Is there any reason the IO package isn't installed here? It seems to be mentioned quite often, but we don't provide a means to install it.
I'm currently going for the minimal thing: get gap_packages spkg working the way it was. We never had io package there, as it was breaking libgap in a bad way. (there are open tickets about io package).
Does this sound like a good plan to you? We can postpone io package, it's not needed to be ready by the Debian freeze deadline.
I need to adjust formatting in a dozen of tests, after that should be ready for review.
OK regarding the IO package. If we weren't installing it in the first place I'm fine just sticking to the minimum set of packages that we were always installing in the first place.
I am also making some updates to this: In particular I'm not happy with the way the compiled packages are installed. Let me know when you're done your changes (set needs_review) and I'll add my changes on top of that.
I'm getting an unpleasant surprise from the doctest framework here - it mangles long lines in this file, e.g. here:
File "src/sage/algebras/quantum_groups/quantum_group_gap.py", line 757, in sage.algebras.quantum_groups.quantum_group_gap.QuantumGroup.coproduct
Failed example:
[Q.coproduct(f, 2)._latex_() for f in Q.F_simple()] # optional - gap_packages
Expected:
['1 {(1<x>1<x>F_{\\alpha_{1}})}+-q^{2}+q^{-2} {(1<x>F_{\\alpha_{1}}<x>[ K_{1} ; 1 ])}+1 {(1<x>F_{\\alpha_{1}}<x>K_{1})}+q^{4}-2+q^{-4} {(F_{\\alpha_{1}}<x>[ K_{1} ; 1 ]<x>[ K_{1} ; 1 ])}+-q^{2}+q^{-2} {(F_{\\alpha_{1}}<x>[ K_{1} ; 1 ]<x>K_{1})}+-q^{2}+q^{-2} {(F_{\\alpha_{1}}<x>K_{1}<x>[ K_{1} ; 1 ])}+1 {(F_{\\alpha_{1}}<x>K_{1}<x>K_{1})}',
'1 {(1<x>1<x>F_{\\alpha_{2}})}+-q+q^{-1} {(1<x>F_{\\alpha_{2}}<x>[ K_{2} ; 1 ])}+1 {(1<x>F_{\\alpha_{2}}<x>K_{2})}+q^{2}-2+q^{-2} {(F_{\\alpha_{2}}<x>[ K_{2} ; 1 ]<x>[ K_{2} ; 1 ])}+-q+q^{-1} {(F_{\\alpha_{2}}<x>[ K_{2} ; 1 ]<x>K_{2})}+-q+q^{-1} {(F_{\\alpha_{2}}<x>K_{2}<x>[ K_{2} ; 1 ])}+1 {(F_{\\alpha_{2}}<x>K_{2}<x>K_{2})}']
Got:
['1 {(1<x>1<x>F_{\\alpha_{1}})}+-q^{2}+q^{-2} {(1<x>F_{\\alpha_{1}}<x>[ K_{1} ; 1 ])}+1 {(1<x>F_{\\alpha_{1}}<x>K_{1})}+q^{4}-2+q^{-4} {(F\n1<x>[ K_{1} ; 1 ]<x>[ K_{1} ; 1 ])}+-q^{2}+q^{-2} {(F_{\\alpha_{1}}<x>[ K_{1} ; 1 ]<x>K_{1})}+-q^{2}+q^{-2} {(F\n1<x>K_{1}<x>[ K_{1} ; 1 ])}+1 {(F_{\\alpha_{1}}<x>K_{1}<x>K_{1})}',
'1 {(1<x>1<x>F_{\\alpha_{2}})}+-q+q^{-1} {(1<x>F_{\\alpha_{2}}<x>[ K_{2} ; 1 ])}+1 {(1<x>F_{\\alpha_{2}}<x>K_{2})}+q^{2}-2+q^{-2} {(F\n4<x>[ K_{2} ; 1 ]<x>[ K_{2} ; 1 ])}+-q+q^{-1} {(F_{\\alpha_{2}}<x>[ K_{2} ; 1 ]<x>K_{2})}+-q+q^{-1} {(F_{\\alpha_{2}}<x>K\n2<x>[ K_{2} ; 1 ])}+1 {(F_{\\alpha_{2}}<x>K_{2}<x>K_{2})}']
and instead of F_{\\alpha_{1}}
at few places it outputs F\n1
- probably some unfortunate combination of escape sequences, dunno... (needless to say, the output at the sage prompt is fine).
I'm inclined to mark these as # known bug
and deal with them later...
In the case of the "grape" package, ISTM the only actual "compiled" part is not technically part of the package at all, but rather it builds its own copy of nauty. It ought to be able to use the nauty that's already included in Sage.
I probably won't stress about it too much in the short time that we have, but this should probably be fixed later.
I agree about nauty/grape - see #26923
Replying to @dimpase:
I'm getting an unpleasant surprise from the doctest framework here - it mangles long lines in this file, e.g. here: and instead of
F_{\\alpha_{1}}
at few places it outputsF\n1
- probably some unfortunate combination of escape sequences, dunno... (needless to say, the output at the sage prompt is fine).
I'm not so sure about that. I don't think the doctest framework is doing that, but I'll look into it later.
Replying to @embray:
Replying to @dimpase:
I'm getting an unpleasant surprise from the doctest framework here - it mangles long lines in this file, e.g. here: and instead of
F_{\\alpha_{1}}
at few places it outputsF\n1
- probably some unfortunate combination of escape sequences, dunno... (needless to say, the output at the sage prompt is fine).I'm not so sure about that. I don't think the doctest framework is doing that, but I'll look into it later.
Might be a "noisy line" thing - but why only while doctesting? Anyhow I have opened #26924 to deal with it.
I'm running the tests for some of these packages. A bunch of tests fail for guava: I'm not sure whether or not this is expected.
A new failure I'm getting is:
File "src/sage/graphs/strongly_regular_db.pyx", line 2445, in sage.graphs.strongly_regular_db.SRG_416_100_36_20
Failed example:
g = SRG_416_100_36_20() # optional - gap_packages # long time
Expected nothing
Got:
#I io package is not available. Check that the name is correct
#I and it is present in one of the GAP root directories (see '??RootPaths')
#I io package is not available. Check that the name is correct
#I and it is present in one of the GAP root directories (see '??RootPaths')
**********************************************************************
1 item had failures:
1 of 4 in sage.graphs.strongly_regular_db.SRG_416_100_36_20
[334 tests, 1 failure, 61.16 s]
So perhaps one of these packages at least tries to load the IO package if available, though doesn't strictly need it. In that case we should at least be silencing this message.
Another one that's a bit troubling is:
sage -t --long src/sage/tests/gap_packages.py
**********************************************************************
File "src/sage/tests/gap_packages.py", line 8, in sage.tests.gap_packages
Failed example:
test_packages(pkgs, only_failures=True) # optional - gap_packages
Expected:
Status Package GAP Output
+--------+---------+------------+
Got:
#I polymake command not found. Please set POLYMAKE_COMMAND by hand
#I HAP warning: Set POLYMAKE_PATH manually if needed.
#I HAP warning: Set BROWSER_PATH manually if needed.
#I HAP warning: Set ASY_PATH manually if needed.
#I You may wish to install the xgap package
#I and enjoy the graphic capabilities of SONATA.
#I qpa-version package is not available. Check that the name is correct
#I and it is present in one of the GAP root directories (see '??RootPaths')
Status Package GAP Output
+---------+-------------+------------+
Failure polenta fail
Failure qpa-version fail
Failure resclasses fail
**********************************************************************
1 item had failures:
1 of 5 in sage.tests.gap_packages
[10 tests, 1 failure, 3.98 s]
I get these warnings when loading the relevant GAP packages in a vanilla GAP source tree, so I suppose they're normal and/or expected, or at least not indicative of any specific bug on our part. But we probably need to at least temporarily silence them for the purposes of the code and tests they affect.
The one exception being the error about "qpa-version". The name of the package is just "qpa", but its source directory is unusually named "qpa-version-<x.y>", so there's some code that's making an invalid assumption about GAP package names :sigh:
Replying to @embray:
A new failure I'm getting is:
File "src/sage/graphs/strongly_regular_db.pyx", line 2445, in sage.graphs.strongly_regular_db.SRG_416_100_36_20 Failed example: g = SRG_416_100_36_20() # optional - gap_packages # long time Expected nothing Got: #I io package is not available. Check that the name is correct #I and it is present in one of the GAP root directories (see '??RootPaths') #I io package is not available. Check that the name is correct #I and it is present in one of the GAP root directories (see '??RootPaths') ********************************************************************** 1 item had failures: 1 of 4 in sage.graphs.strongly_regular_db.SRG_416_100_36_20 [334 tests, 1 failure, 61.16 s]
So perhaps one of these packages at least tries to load the IO package if available, though doesn't strictly need it. In that case we should at least be silencing this message.
Yes, atlasrep does this. However, you might have a stale libgap workspace. I saw this warning, and then went away after I wiped the workspaces out.
Replying to @embray:
Another one that's a bit troubling is:
sage -t --long src/sage/tests/gap_packages.py ********************************************************************** File "src/sage/tests/gap_packages.py", line 8, in sage.tests.gap_packages Failed example: test_packages(pkgs, only_failures=True) # optional - gap_packages Expected: Status Package GAP Output +--------+---------+------------+ Got: #I polymake command not found. Please set POLYMAKE_COMMAND by hand #I HAP warning: Set POLYMAKE_PATH manually if needed. #I HAP warning: Set BROWSER_PATH manually if needed. #I HAP warning: Set ASY_PATH manually if needed. #I You may wish to install the xgap package #I and enjoy the graphic capabilities of SONATA. #I qpa-version package is not available. Check that the name is correct #I and it is present in one of the GAP root directories (see '??RootPaths') Status Package GAP Output +---------+-------------+------------+ Failure polenta fail Failure qpa-version fail Failure resclasses fail ********************************************************************** 1 item had failures: 1 of 5 in sage.tests.gap_packages [10 tests, 1 failure, 3.98 s]
I get these warnings when loading the relevant GAP packages in a vanilla GAP source tree, so I suppose they're normal and/or expected, or at least not indicative of any specific bug on our part. But we probably need to at least temporarily silence them for the purposes of the code and tests they affect.
The one exception being the error about "qpa-version". The name of the package is just "qpa", but its source directory is unusually named "qpa-version-<x.y>", so there's some code that's making an invalid assumption about GAP package names :sigh:
Yes, that's where I am now too. I'll need to step away for an hour, I'll push what I have in a second.
OK, over to you now.
I think this change
diff --git a/src/sage/algebras/quantum_groups/quantum_group_gap.py b/src/sage/algebras/quantum_groups/quantum_group_gap.py
index 4ce98f4..0e4549b 100644
--- a/src/sage/algebras/quantum_groups/quantum_group_gap.py
+++ b/src/sage/algebras/quantum_groups/quantum_group_gap.py
@@ -1566,8 +1564,8 @@ class QuantumGroupModule(Parent, UniqueRepresentation):
sage: Q = QuantumGroup(['A',2]) # optional - gap_packages
sage: V = Q.highest_weight_module([1,1]) # optional - gap_packages
sage: V.gap() # optional - gap_packages
- <8-dimensional left-module over QuantumUEA( <root system of type A2>,
- Qpar = q )>
+ <8-dimensional left-module over QuantumUEA( <root system of type A
+ 2>, Qpar = q )>
"""
return self._libgap
This one feels like a bug in the GAP and/or the QuaGroup package to me, but I confirmed that this is in fact the output provided by GAP and not a bug on our end, so for the time being there's nothing to be done (other than possibly report it upstream)
This is one of the problems with GAP objects not having a real string representation that isn't tied to how exactly they're printed to the screen :|
The problem here, accounting for all these test failures including the test above that was modified, is that the QuaGroups package seems (I have not looked for the exact code) to insert linebreaks into the representations of its types in different places depending on the terminal width. Not exactly sure if that's a GAP thing in general or something specific to this package, but this is the first place it's come up. In any case we apparently aren't dealing well with those unnecessary line breaks.
linebreaks are inserted by GAP, it was always the case.
I see now, the output splitting is indeed done by default by GAP. Again, this is no good, especially since it can depend on terminal width. I haven't found yet exactly where GAP determines this, but I need to figure out a way to force it to just not do this at all, if possible. That might be something to do as part of #22626 or as a follow-up.
Branch pushed to git repo; I updated commit sha1. New commits:
fc19792 | install deps of polenta, qpa, resclasses, fix test |
Replying to @embray:
Another one that's a bit troubling is:
sage -t --long src/sage/tests/gap_packages.py [.....] +---------+-------------+------------+ Failure polenta fail Failure qpa-version fail Failure resclasses fail ********************************************************************** 1 item had failures: 1 of 5 in sage.tests.gap_packages [10 tests, 1 failure, 3.98 s]
I get these warnings when loading the relevant GAP packages in a vanilla GAP source tree, so I suppose they're normal and/or expected, or at least not indicative of any specific bug on our part. But we probably need to at least temporarily silence them for the purposes of the code and tests they affect.
The one exception being the error about "qpa-version". The name of the package is just "qpa", but its source directory is unusually named "qpa-version-<x.y>", so there's some code that's making an invalid assumption about GAP package names :sigh:
the latest commit fixes these failures, which are two-fold:
missing dependencies and weird -version
-part which one should strip.
I should still figure out how to suppress these #I ...
lines...
Branch pushed to git repo; I updated commit sha1. New commits:
92517b4 | suppress "#I.." warnings |
Branch pushed to git repo; I updated commit sha1. New commits:
08ffb33 | more doctest fixes |
By the way, I tried installing IO package in the same way as this ticket builds packages needing compilation, but this fails due to a "missing" header (not sure what goes wrong):
[gap_packages-4.10.0] configure: creating ./config.status
[gap_packages-4.10.0] config.status: creating Makefile
[gap_packages-4.10.0] config.status: creating src/pkgconfig.h
[gap_packages-4.10.0] config.status: executing depfiles commands
[gap_packages-4.10.0] config.status: executing libtool commands
[gap_packages-4.10.0] make[2]: Entering directory '/mnt/opt/Sage/sage-dev/local/share/gap/pkg/io-4.5.4'
[gap_packages-4.10.0] /bin/sh ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I./src -I/mnt/opt/Sage/sage-dev/local/var/tmp/sage/build/gap-4.10.0/src/gen -I/mnt/opt/Sage/sage-dev/local/var/tmp/sage/build/gap-4.10.0/src/src -I/mnt/opt/Sage/sage-dev/local/var/tmp/sage/build/gap-4.10.0/src -DHAVE_CONFIG_H -I/mnt/opt/Sage/sage-dev/local/include -O2 -g -g -O2 -MT src/io_la-io.lo -MD -MP -MF src/.deps/io_la-io.Tpo -c -o src/io_la-io.lo `test -f 'src/io.c' || echo './'`src/io.c
[gap_packages-4.10.0] libtool: compile: gcc -DHAVE_CONFIG_H -I. -I./src -I/mnt/opt/Sage/sage-dev/local/var/tmp/sage/build/gap-4.10.0/src/gen -I/mnt/opt/Sage/sage-dev/local/var/tmp/sage/build/gap-4.10.0/src/src -I/mnt/opt/Sage/sage-dev/local/var/tmp/sage/build/gap-4.10.0/src -DHAVE_CONFIG_H -I/mnt/opt/Sage/sage-dev/local/include -O2 -g -g -O2 -MT src/io_la-io.lo -MD -MP -MF src/.deps/io_la-io.Tpo -c src/io.c -fPIC -DPIC -o src/.libs/io_la-io.o
[gap_packages-4.10.0] src/io.c:14:10: fatal error: src/compiled.h: No such file or directory
[gap_packages-4.10.0] #include "src/compiled.h" /* GAP headers */
[gap_packages-4.10.0] ^~~~~~~~~~~~~~~~
looks like the corresponding -I
arguments all have an extra /src
suffix, something that prevents this header to be found.
..."thanks" to Gatwick drone operators, I can do this work now, and not sit on a plane off to Singapore...
GAP's IO is on #26930
Branch pushed to git repo; I updated commit sha1. New commits:
c2e30ae | silence one more #I message from GAP |
OK, seems to be ready now
diff --git a/src/sage/graphs/strongly_regular_db.pyx b/src/sage/graphs/strongly_regular_db.pyx
index 8df9bf5..06a0780 100644
--- a/src/sage/graphs/strongly_regular_db.pyx
+++ b/src/sage/graphs/strongly_regular_db.pyx
@@ -2442,7 +2442,9 @@ def SRG_416_100_36_20():
EXAMPLES::
sage: from sage.graphs.strongly_regular_db import SRG_416_100_36_20
+ sage: libgap.eval("SetInfoLevel(InfoWarning,0)") # silence #I warnings
sage: g = SRG_416_100_36_20() # optional - gap_packages # long time
+ sage: libgap.eval("SetInfoLevel(InfoWarning,1)") # silence #I warnings
sage: g.is_strongly_regular(parameters=True) # optional - gap_packages # long time
(416, 100, 36, 20)
"""
I don't like that this is done in the doctest; it distracts from the actual documentation. I think this should go in the function itself. We've already seen the warning in question, decided it is not important to us, and can go ahead and silence it.
Replying to @sagetrac-git:
Branch pushed to git repo; I updated commit sha1. New commits:
08ffb33
more doctest fixes
This change doesn't seem to work for me. Those tests passed before this commit, and fail after. I tried clearing my existing workspaces but it still fails. That's troubling--it may still be some badly managed random state somewhere. Unless I'm missing something else.
Replying to @dimpase:
..."thanks" to Gatwick drone operators, I can do this work now, and not sit on a plane off to Singapore...
Huh?? What happened? :(
Replying to @embray:
Replying to @dimpase:
..."thanks" to Gatwick drone operators, I can do this work now, and not sit on a plane off to Singapore...
Huh?? What happened? :(
Lucky you, you don't read news. :-) Gatwick was shut down for ~36 hours due to some not yet caught folks flying a largish drone over it several times, 700+ flights cancelled, 100s diverted. We were able to book an even cheaper flight, via Berlin, and get a full refund. They are open since this morning, and in fact the airline rebooked us for tonight, but we were in the full panic mode and got the other flight already...
Replying to @embray:
Replying to @sagetrac-git:
Branch pushed to git repo; I updated commit sha1. New commits:
08ffb33 more doctest fixes
This change doesn't seem to work for me. Those tests passed before this commit, and fail after. I tried clearing my existing workspaces but it still fails. That's troubling--it may still be some badly managed random state somewhere. Unless I'm missing something else.
they pass for me, both with and without --long
, on Linux and on OSX.
I'd try rm -rf local/share/gap local/include/gap
followed by ./sage -f gap gap_packages
followed up by make build
, and then testing...
(and wiping out ~/.sage/gap/
too)
Branch pushed to git repo; I updated commit sha1. New commits:
1e5f679 | move silencing into the function body |
Replying to @sagetrac-git:
Branch pushed to git repo; I updated commit sha1. New commits:
1e5f679
move silencing into the function body
this addresses comment 65.
Replying to @embray:
Replying to @sagetrac-git:
Branch pushed to git repo; I updated commit sha1. New commits:
08ffb33 more doctest fixes
This change doesn't seem to work for me. Those tests passed before this commit, and fail after. I tried clearing my existing workspaces but it still fails. That's troubling--it may still be some badly managed random state somewhere. Unless I'm missing something else.
Nevermind, that was my fault. Nevertheless, this still won't work: The outcomes of these tests depend on whether or not gap_packages
is installed, so if it isn't then these will give different results. I'm not sure what to do: Maybe these tests should be rewritten to be more flexible, or the code that's being tested needs to be rewritten to be less sensitive to what GAP packages are installed (if that's even possible...)
Let's just make gap_packages standard, as planned anyway. This will alleviate this problem.
After the upgrade to GAP 4.10 in #22626, we revisit what GAP packages are available in Sage via the GAP standard SPKG and via extra optional SPGKs.
All GAP packages formerly in our "database_gap" optional SPKG, except tomlib, now get installed with GAP as part of our "gap" standard SPKG.
This ticket adds tomlib to the "gap" standard SPKG and removes the "database_gap" optional SPKG.
(All packages previously in "database_gap" have GPL-compatible licenses and can therefore be distributed as part of the "gap" standard SPKG without license worries.)
We also update the list of extra GAP packages in gap_packages, which remains a separate and optional SPKG.
Once the present ticket is merged, GAP packages shipped with the "gap" standard SPKG and with the "gap_packages" optional SPKG are as follows.
'''GAP packages now shipped with "gap" (standard SageMath package)'''
A basic package used by most GAP packages for their documentation
Five database packages formerly in "database_gap"
Twelve more packages
'''GAP packages now shipped with "gap_packages" (optional SageMath package)'''
Twenty-five "pure GAP" packages (not requiring compilation)
Four compiled GAP packages
No longer distributed
Depends on #22626 Depends on #26889 Depends on #26965
CC: @alex-konovalov @antonio-rojas @dimpase @embray @kiwifb @mantepse @miguelmarco @slel @tscrim @vbraun
Component: packages: optional
Keywords: GAP
Author: Dima Pasechnik, Erik Bray
Branch:
8bf1044
Reviewer: Erik Bray, Dima Pasechnik
Issue created by migration from https://trac.sagemath.org/ticket/26856