Closed nthiery closed 5 years ago
Replying to @dimpase:
there is also libgap unprefixing in Python/Cython that has to be done. Perhaps separating these from C level unprefixing would help making it smoother.
Yes, that is what I meant. Creating a separate ticket for unprefixing only at the Cython level by using macros to translate between the unprefixed names in Cython and the prefixed names in libGAP.
Replying to @jdemeyer:
Replying to @dimpase:
there is also libgap unprefixing in Python/Cython that has to be done. Perhaps separating these from C level unprefixing would help making it smoother.
Yes, that is what I meant. Creating a separate ticket for unprefixing only at the Cython level by using macros to translate between the unprefixed names in Cython and the prefixed names in libGAP.
I am not sure I understand. Currently in Cython on can do something like
from sage.libs.gap.libgap import libgap
g=libgap.ProjectiveGeneralLinearGroup(3,3)
How will this change under what you propose?
Replying to @dimpase:
from sage.libs.gap.libgap import libgap g=libgap.ProjectiveGeneralLinearGroup(3,3)
How will this change under what you propose?
The Python interface won't change at all. I am talking about how sage.libs.gap
calls libGAP.
As far as I can tell, GAP 4.9 has not been released. Does anybody know how close we are to an actual release?
I think GAP 4.9.0 is considered a public beta for GAP 4.9.
The list of releases at
still displays GAP 4.8.10 as the latest release.
In the list of past releases there, the GAP 4.8, 4.7, 4.6, 4.5 series start at GAP 4.8.2, 4.7.2, 4.6.2, 4.5.4, so probably GAP 4.x.y with those x and lower y were considered beta.
Still, seeing how 4.9.0 works with Sage would be nice.
Dependencies: #25273
Where is the GAP source tarball which is supposed to be used here?
Changed branch from u/nthiery/upgrade_to_gap_4_9 to u/jdemeyer/upgrade_to_gap_4_9
Replying to @jdemeyer:
Where is the GAP source tarball which is supposed to be used here?
Description changed:
---
+++
@@ -3,7 +3,7 @@
The branch attached to this ticket updates Sage to run on top of
[a branch of GAP](https://github.com/markuspf/gap/tree/hpc-merge-libgap)
-by Markus Pfeiffer that adds libgap compilation and [will be merged](https://github.com/fingolfin/gap/pull/64) soon in the devel version of GAP.
+by Markus Pfeiffer that adds libgap compilation and [will be merged](https://github.com/gap-system/gap/pull/1205) soon in the devel version of GAP.
See https://github.com/markuspf/gap/issues/2 for the few sticking points that could prevent using a vanilla GAP from the distribution (please edit further if you think about more of them).
@@ -13,7 +13,7 @@
- Update the gap spkg to the new build system and build and install libgap
-- Replace gap.shi.patch by a plain gap startup script for Sage
+- Replace `gap.shi.patch` by a plain gap startup script for Sage
Rationale: GAP used to provide a startup shell script. The GAP devs
are in the process of getting rid of it and provide a very minimal
@@ -22,7 +22,7 @@
- Update a few doctests w.r.t. changes of output of some GAP functions
- **Possibly controversial:** The new libgap currently *does not come*
- with symbol rewriting (Foo -> libGAP_Foo). This avoids messing
+ with symbol rewriting (`foobar` -> `libGAP_foobar`). This avoids messing
around with GAP's sources; in particular opening the door for using
a stock GAP from the OS distribution. However there always is a risk
of name conflict. And indeed, GAP's constants (actually cpp macros)
@@ -51,13 +51,10 @@
TODO:
- Automatic handling of headers (see below for how to do it by hand).
- GAP's build system will eventuall provide a rule to install headers
+ GAP's build system will eventually provide a rule to install headers
which will make this trivial.
- Use GAP's own `make install` [when it will be implemented](https://github.com/fingolfin/gap/issues/44).
-
-- Update the documentation in sage.libs.gap.libgap.pyx to not mention
- the `libgap_` prefix
- Check against #19915 to see if any of the changes there should be
ported here. Then close as won't fix.
@@ -162,4 +159,3 @@
Replying to @nthiery:
The branch attached to this ticket updates Sage to run on top of a branch of GAP by Markus Pfeiffer that adds libgap compilation and will be merged soon in the devel version of GAP.
It seems that this branch which "will be merged soon" is still not merged. It would be good to have an idea of whether or not that will happen.
According to https://github.com/gap-system/gap/pull/1205#issuecomment-385670578, Markus's libgap branch won't be merged in GAP 4.9: it's not yet ready enough. Bummer.
We are discussing with Sebastian and Thomas what we can do to move forward here. They are very interested as well by libgap for the integration between GAP and Julia.
Tentative steps:
GAP 4.9.1 was released in May 2018.
This is the first stable release of GAP 4.9 (GAP 4.9.0 was a beta for GAP 4.9).
Description changed:
---
+++
@@ -1,11 +1,18 @@
-GAP 4.9 will come with a completely rewritten build system that will
-simplify our packaging. In fact, it may well enable Sage to use a vanilla GAP installation as provided by the distribution. It will also enable building GAP as a library, so that we can get rid of our separate libgap package.
+GAP 4.9 comes with a completely rewritten build system that will simplify
+our packaging. In fact, it may well enable Sage to use a vanilla GAP
+installation as provided by the distribution.
+
+The GAP team will not provide libGAP with GAP 4.9, but only with GAP 4.10.
The branch attached to this ticket updates Sage to run on top of
[a branch of GAP](https://github.com/markuspf/gap/tree/hpc-merge-libgap)
-by Markus Pfeiffer that adds libgap compilation and [will be merged](https://github.com/gap-system/gap/pull/1205) soon in the devel version of GAP.
+by Markus Pfeiffer that adds libgap compilation and
+[might be merged](https://github.com/gap-system/gap/pull/1205)
+soon in the devel version of GAP.
-See https://github.com/markuspf/gap/issues/2 for the few sticking points that could prevent using a vanilla GAP from the distribution (please edit further if you think about more of them).
+See https://github.com/markuspf/gap/issues/2 for the few sticking points
+that could prevent using a vanilla GAP from the distribution (please edit
+further if you think about more of them).
What the branch does:
@@ -159,3 +166,8 @@
This should be fixed once GAP's gap binary is built on top of libgap. See: https://github.com/markuspf/gap/issues/1. + +Note: + +- GAP 4.9.1 was released in May 2018.
I'm confused - I just built GAP 4.9.1 from the upstream tarball and 'make install' does install /usr/lib/libgap.so and libgap's headers. Is this a different libgap than the one mentioned here to be included in 4.10?
One comment at pull request 1205 in GAP's github repo said GAP won't ship libGAP before GAP 4.10:
Maybe this was changed though.
Description changed:
---
+++
@@ -171,3 +171,4 @@
- [GAP 4.9.1 was released](https://mail.gap-system.org/pipermail/forum/2018/005741.html) in May 2018.
This is the first stable release of GAP 4.9 (GAP 4.9.0 was a beta for GAP 4.9).
+- Max Horn reviewed the list of GAP symbols we use in Sage; some have already changed in 4.9. See this [pad](https://hackmd.io/emNi76svSWCh1fBeLKqPdA?edit#) for notes.
Has anybody asked GAP developers for a policy on the prefixing?
In Cernay, I proposed that GAP should prefix some/all of their names like T_INT
-> GAP_T_INT
. For compiling GAP (+ GAP packages) itself, we can use a file with defines like
#define T_INT GAP_T_INT
to ensure backwards compatibility. This way, in GAP itself, one could use either T_INT
or GAP_T_INT
. Outside of GAP, only GAP_T_INT
would work.
I think this is relatively easy to implement (I'm willing to do it) but I don't know what upstream GAP thinks of it.
Description changed:
---
+++
@@ -169,6 +169,7 @@
Note:
-- [GAP 4.9.1 was released](https://mail.gap-system.org/pipermail/forum/2018/005741.html) in May 2018.
- This is the first stable release of GAP 4.9 (GAP 4.9.0 was a beta for GAP 4.9).
+- [GAP 4.9.2](http://www.gap-system.org/Releases/4.9.2.html) was released in July 2018.
+- [GAP 4.9.1](http://www.gap-system.org/Releases/4.9.1.html), released in May 2018,
+ was the first stable release of GAP 4.9 (GAP 4.9.0 was a beta for GAP 4.9).
- Max Horn reviewed the list of GAP symbols we use in Sage; some have already changed in 4.9. See this [pad](https://hackmd.io/emNi76svSWCh1fBeLKqPdA?edit#) for notes.
Hi,
For the record: I experimented a bit a month ago and a bit more
this morning with Dima with the latest gap master that
has make libgap
and make install-libgap
rules.
See the following branch, which builds libgap from GAP's pristine
sources:
https://github.com/sagemath/sagetrac-mirror/commits/u/nthiery/gap-libgap
Note that, at this stage, Sage won't start with it; maybe even not compile.
Has anybody asked GAP developers for a policy on the prefixing?
In Cernay, I proposed that GAP should prefix some/all of their names like
T_INT
->GAP_T_INT
. For compiling GAP (+ GAP packages) itself, we can use a file with defines like#define T_INT GAP_T_INT
to ensure backwards compatibility. This way, in GAP itself, one could use either
T_INT
orGAP_T_INT
. Outside of GAP, onlyGAP_T_INT
would work.I think this is relatively easy to implement (I'm willing to do it) but I don't know what upstream GAP thinks of it.
That's the idea indeed. Eventually they will provide a standalone header file that will define the official GAP API, and where all symbols will be prefixed. To bootstrap the process, we (well mostly Max!) reviewed and annotated the symbols that are used in Sage:
https://hackmd.io/emNi76svSWCh1fBeLKqPdA#
Thank you for working on this! Sage using an up-to-date gap and doing away with the additional libgap package will simplify packaging.
update milestone 8.3 -> 8.4
Changed work issues from Wait for gap 4.9 release to Wait for gap 4.10 release
Description changed:
---
+++
@@ -2,7 +2,7 @@
our packaging. In fact, it may well enable Sage to use a vanilla GAP
installation as provided by the distribution.
-The GAP team will not provide libGAP with GAP 4.9, but only with GAP 4.10.
+The GAP team will provide libGAP with GAP 4.10.
The branch attached to this ticket updates Sage to run on top of
[a branch of GAP](https://github.com/markuspf/gap/tree/hpc-merge-libgap)
Description changed:
---
+++
@@ -1,18 +1,14 @@
-GAP 4.9 comes with a completely rewritten build system that will simplify
+GAP 4.9 came with a completely rewritten build system that will simplify
our packaging. In fact, it may well enable Sage to use a vanilla GAP
installation as provided by the distribution.
The GAP team will provide libGAP with GAP 4.10.
-The branch attached to this ticket updates Sage to run on top of
-[a branch of GAP](https://github.com/markuspf/gap/tree/hpc-merge-libgap)
-by Markus Pfeiffer that adds libgap compilation and
-[might be merged](https://github.com/gap-system/gap/pull/1205)
-soon in the devel version of GAP.
+The branch attached to this ticket updates Sage to run on top of the current GAP dev branch. (the branch of GAP
+by Markus Pfeiffer that adds libgap compilation has been merged).
-See https://github.com/markuspf/gap/issues/2 for the few sticking points
-that could prevent using a vanilla GAP from the distribution (please edit
-further if you think about more of them).
+Some historical notes may be found in https://github.com/markuspf/gap/issues/2
+
What the branch does:
@@ -53,7 +49,7 @@
can explain, e.g. change of orders in lists of elements. So those
should be nothing to worry about. There is not much point in updating
those doctests right away; we may as well wait for a more final
-version of 4.9 to be out.
+version of 4.10 to be out.
TODO:
@@ -70,14 +66,11 @@
- ???
-Fetching Markus's GAP sources:
+Fetching GAP sources:
-git clone git@github.com:markuspf/gap.git $LIBGAP +git clone git@github.com:gap-system/gap.git $LIBGAP cd $LIBGAP -git remote add markuspf git@github.com:markuspf/gap.git -git fetch markuspf -git checkout -b markuspf/hpc-merge-libgap ./autogen.sh ./configure make bootstrap-pkg-minimal @@ -87,9 +80,9 @@ Testing libgap:
-./configure --enable-libgap
-make -j4 libgap
-make test-libgap
+./configure
+make -j4
+make testlibgap
Build and install a tardist for Sage, and rebuild the spkg: @@ -101,7 +94,7 @@ make manuals make clean
-(cd ..; tar zcvf $SAGE/upstream/$GAP.tar.gz --exclude .git $GAP) +(cd ..; mv $GAP src; tar zcvf $SAGE/upstream/$GAP.tar.gz --exclude .git src)
sage --package fix-checksum sage -f gap # -s
Description changed:
---
+++
@@ -1,14 +1,18 @@
-GAP 4.9 came with a completely rewritten build system that will simplify
+GAP 4.9 comes with a completely rewritten build system that will simplify
our packaging. In fact, it may well enable Sage to use a vanilla GAP
installation as provided by the distribution.
The GAP team will provide libGAP with GAP 4.10.
-The branch attached to this ticket updates Sage to run on top of the current GAP dev branch. (the branch of GAP
-by Markus Pfeiffer that adds libgap compilation has been merged).
+The branch attached to this ticket updates Sage to run on top of
+[a branch of GAP](https://github.com/markuspf/gap/tree/hpc-merge-libgap)
+by Markus Pfeiffer that adds libgap compilation and
+[might be merged](https://github.com/gap-system/gap/pull/1205)
+soon in the devel version of GAP.
-Some historical notes may be found in https://github.com/markuspf/gap/issues/2
-
+See https://github.com/markuspf/gap/issues/2 for the few sticking points
+that could prevent using a vanilla GAP from the distribution (please edit
+further if you think about more of them).
What the branch does:
@@ -49,7 +53,7 @@
can explain, e.g. change of orders in lists of elements. So those
should be nothing to worry about. There is not much point in updating
those doctests right away; we may as well wait for a more final
-version of 4.10 to be out.
+version of 4.9 to be out.
TODO:
@@ -66,11 +70,14 @@
- ???
-Fetching GAP sources:
+Fetching Markus's GAP sources:
-git clone git@github.com:gap-system/gap.git $LIBGAP +git clone git@github.com:markuspf/gap.git $LIBGAP cd $LIBGAP +git remote add markuspf git@github.com:markuspf/gap.git +git fetch markuspf +git checkout -b markuspf/hpc-merge-libgap ./autogen.sh ./configure make bootstrap-pkg-minimal @@ -80,9 +87,9 @@ Testing libgap:
-./configure
-make -j4
-make testlibgap
+./configure --enable-libgap
+make -j4 libgap
+make test-libgap
Build and install a tardist for Sage, and rebuild the spkg: @@ -94,7 +101,7 @@ make manuals make clean
-(cd ..; mv $GAP src; tar zcvf $SAGE/upstream/$GAP.tar.gz --exclude .git src) +(cd ..; tar zcvf $SAGE/upstream/$GAP.tar.gz --exclude .git $GAP)
sage --package fix-checksum sage -f gap # -s
note that the prefix of the tarball should start with src/
- else one gets weird error messages from the python's tar.
GAPs make install
target now works in a preliminary form - on Linux for sure.
Changed branch from u/jdemeyer/upgrade_to_gap_4_9 to u/dimpase/WIP/libgap410
This is WIP - builds, but libgap.pyx interaction loop has to be rewritten using new API.
Also, sage --gap
is broken
gap: hmm, I cannot find 'lib/init.g' maybe use option '-l <gaproot>'?
New commits:
c766ad6 | Merge branch 'u/jdemeyer/upgrade_to_gap_4_9' of trac.sagemath.org:sage into libgap |
5152637 | building GAP and libGAP from GAP |
6bed83a | changes for using libGAP from GAP - part 1 |
f069122 | removing util* |
oops, one still needs initialize() from util.pyx, in some form...
Great to see the progress! Sounds like a productive week in Siegen. Thanks so much.
Description changed:
---
+++
@@ -173,3 +173,5 @@
- [GAP 4.9.1](http://www.gap-system.org/Releases/4.9.1.html), released in May 2018,
was the first stable release of GAP 4.9 (GAP 4.9.0 was a beta for GAP 4.9).
- Max Horn reviewed the list of GAP symbols we use in Sage; some have already changed in 4.9. See this [pad](https://hackmd.io/emNi76svSWCh1fBeLKqPdA?edit#) for notes.
+
+Tarball : https://github.com/dimpase/gap/releases/download/dev/gap-4.dev.tar.gz
Branch pushed to git repo; I updated commit sha1. New commits:
14919d3 | update util and the GAP package |
Branch pushed to git repo; I updated commit sha1. New commits:
a0616de | adding --nointeract to libgap startup options |
at this point I am at
sage: libgap(1)
Error, Variable: '_rich_repr_' must have a value
Error, Variable: '_ipython_canary_method_should_not_exist_' must have a\
value
[... several more errors like above]
gap: panic, could not open *errout*!
GAP 4.10 comes with a completely rewritten build system that will simplify our packaging. In particular, libGAP no longer needs to be a separate package.
What the branch does:
Remove the libgap spkg
Update the gap spkg to the new build system and build and install libgap
Replace
gap.shi.patch
by a plain gap startup script for SageRationale: GAP used to provide a startup shell script. The GAP devs are in the process of getting rid of it and provide a very minimal one. They recommend to just write our own rather than patching it.
Update a few doctests w.r.t. changes of output of some GAP functions
Reworks how gap is installed in
$SAGE_LOCAL
: Rather than installing the entire source tree we just install thegap
andlibgap
binaries to standard locations, and add a$SAGE_LOCAL/share/gap
containing theGAP_ROOT_DIR
which is stripped down to the minimum needed for GAP to work (standard libs and packages, docs, as well as the source code for introspection of kernel functions, but all build artifacts are carefully omitted).foobar
->libGAP_foobar
). This avoids messing around with GAP's sources; in particular opening the door for using a stock GAP from the OS distribution. However there always is a risk of name conflicts. And indeed, GAP's enumsT_INT
,T_FLOAT
, ... conflict with Python's constants defined instructmember.h
. This is hopefully not actually a problem in practice due to the way how Cython orders includes.Something similar was started by Volker at #19915.
Removes configure.patch: it was patching configure for better GMP detection under Cygwin (#13954). This should not be needed anymore with the new build system and use of --with-gmp. If it is, upstream asked for it to be reported and they will fix it.
Removes additional patches for now--we would like to focus on being able to work with a system GAP as much as possible.
Revert #19726 (not needed anymore)
Status
Currently broken - crashes deep inside GAP error handling system after few simple commands.
Basic tests on libgap:
Running most relevant tests:
Current status: lots of errors
Still have some miscellaneous segfaults and other weird crashes
Still have work to do on improving error handling; replacing the built-in
ErrorInner
function might help here.SIGINT handling by Python is broken by
GAP_Intialize
; need to work around this.Testing packages with dynamic loading (e.g. IO):
Install IO:
Test it locally:
This does not yet work:
This should be fixed once GAP's gap binary is built on top of libgap. See: https://github.com/markuspf/gap/issues/1 I believe this is fixed, but there are still some problems with the way this ticket is "installing" GAP for
$SAGE_LOCAL
.Note:
Upstream PRs
A few pull requests we made to improve use of GAP as a library:
Other open issues that don't have PRs yet:
Some other PRs useful to this effort:
The update of gap_packages and database_gap is on #26856
Tarball: https://www.gap-system.org/pub/gap/gap-4.10/tar.gz/gap-4.10.0.tar.gz
Depends on #26874
CC: @alex-konovalov @dimpase @embray @kiwifb @antonio-rojas @sebasguts @jpflori @sagetrac-markuspf @nthiery @slel @vbraun @williamstein @timokau
Component: interfaces
Keywords: days85, libgap
Stopgaps: error handling in libgap, documentation display
Author: Nicolas M. Thiéry, Dima Pasechnik, Erik Bray, Jeroen Demeyer
Branch:
b446ebb
Reviewer: Erik Bray, Dima Pasechnik, Jeroen Demeyer
Issue created by migration from https://trac.sagemath.org/ticket/22626