search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.33k stars 453 forks source link

Upgrade: Singular 4.2.0, pysingular 0.9.7 #25993

Closed jdemeyer closed 3 years ago

jdemeyer commented 6 years ago

Tarball: see checksums.ini on the branch

Use make SAGE_SPKG="sage-spkg -o" singular-clean sagelib-clean build to automatically download and install.

"Critical" because it enables supporting newer versions of FLINT.

We use the Singular development branch (spielwiese) + PR https://github.com/Singular/Singular/pull/1058 in order to build documentation. The tarball is made from https://github.com/mkoeppe/Singular/tree/Release-4-2-0-p1%2Bsage

Upstream: Fixed upstream, but not in a stable release.

CC: @simon-king-jena @timokau @slel @isuruf @saraedum @dkrenn @sagetrac-araichev @cheuberg @behackl @dimpase @videlec

Component: packages: standard

Keywords: upgrade, Singular, pysingular

Author: Antonio Rojas, Markus Wageringel, Matthias Koeppe

Branch: ec471e0

Reviewer: Matthias Koeppe, Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/25993

mwageringel commented 4 years ago
comment:43

With Singular 4.1.2p2, building Pynac fails with errors like this:

In file included from /amd/compute/mwagerin/git/sage_compute/python3/local/include/factory/factory.h:26,
                 from mpoly-singular.cpp:27:
/amd/compute/mwagerin/git/sage_compute/python3/local/include/factory/factoryconf.h:21:10: fatal error: factory/globaldefs.h: No such file or directory
 #include "factory/globaldefs.h"
          ^~~~~~~~~~~~~~~~~~~~~~

I do not know how to resolve this.

mwageringel commented 4 years ago

Description changed:

--- 
+++ 
@@ -1 +1,4 @@
-**Tarball**: https://service.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-1-2/singular-4.1.2p1.tar.gz
+**Tarball 4.1.2p1**: https://service.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-1-2/singular-4.1.2p1.tar.gz
+
+**Tarball 4.1.2p2**: ftp://jim.mathematik.uni-kl.de/pub/Math/Singular/SOURCES/4-1-2/singular-4.1.2p2.tar.gz
+
kiwifb commented 4 years ago
comment:44

Replying to @mwageringel:

With Singular 4.1.2p2, building Pynac fails with errors like this:

In file included from /amd/compute/mwagerin/git/sage_compute/python3/local/include/factory/factory.h:26,
                 from mpoly-singular.cpp:27:
/amd/compute/mwagerin/git/sage_compute/python3/local/include/factory/factoryconf.h:21:10: fatal error: factory/globaldefs.h: No such file or directory
 #include "factory/globaldefs.h"
          ^~~~~~~~~~~~~~~~~~~~~~

I do not know how to resolve this.

Does the file exists? If not it looks like a bug in singular's installation. I'll have a look locally when I can.

antonio-rojas commented 4 years ago
comment:45

Replying to @mwageringel:

With Singular 4.1.2p2, building Pynac fails with errors like this:

In file included from /amd/compute/mwagerin/git/sage_compute/python3/local/include/factory/factory.h:26,
                 from mpoly-singular.cpp:27:
/amd/compute/mwagerin/git/sage_compute/python3/local/include/factory/factoryconf.h:21:10: fatal error: factory/globaldefs.h: No such file or directory
 #include "factory/globaldefs.h"
          ^~~~~~~~~~~~~~~~~~~~~~

I do not know how to resolve this.

This is a bug in singular install: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/singular#n42

kiwifb commented 4 years ago
comment:46

Did you fill a ticket upstream?

antonio-rojas commented 4 years ago
comment:47

Replying to @kiwifb:

Did you fill a ticket upstream?

No - i'm tired of filing singular bug reports just to have them ignored.

mwageringel commented 4 years ago
comment:48

Replying to @antonio-rojas:

This is a bug in singular install: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/singular#n42

Thank you. That works.

antonio-rojas commented 4 years ago
comment:49

Replying to @antonio-rojas:

Replying to @kiwifb:

Did you fill a ticket upstream?

No - i'm tired of filing singular bug reports just to have them ignored.

Seems to be fixed in git master

kiwifb commented 4 years ago
comment:50

Replying to @antonio-rojas:

Replying to @antonio-rojas:

Replying to @kiwifb:

Did you fill a ticket upstream?

No - i'm tired of filing singular bug reports just to have them ignored.

Seems to be fixed in git master

If you are thinking of https://github.com/Singular/Sources/commit/557b878dd8c840afb5ec95de122ba27a0ec99926 it is in my source for 4.1.2p2 and it obviously doesn't help. Are you thinking of another commit?

antonio-rojas commented 4 years ago
comment:51

Replying to @kiwifb:

Replying to @antonio-rojas:

Replying to @antonio-rojas:

Replying to @kiwifb:

Did you fill a ticket upstream?

No - i'm tired of filing singular bug reports just to have them ignored.

Seems to be fixed in git master

If you are thinking of https://github.com/Singular/Sources/commit/557b878dd8c840afb5ec95de122ba27a0ec99926 it is in my source for 4.1.2p2 and it obviously doesn't help. Are you thinking of another commit?

I don't know which commit fixed it, I just compiled git master and checked that the header is installed.

kiwifb commented 4 years ago
comment:52

All right it is actually fixed in the next commit https://github.com/Singular/Sources/commit/10ccdf4ec3a17fc1e6b47d06f9abf4565ae63341

kind of sad.

kiwifb commented 4 years ago
comment:53

I have checked, that commit is all you need to get this particular header.

mwageringel commented 4 years ago

Work Issues: update letterplace algebras

mwageringel commented 4 years ago
comment:54

Regarding the degrees option of letterplace algebras, Singular should support weighted degree orders naturally now, so Sage's workaround using slack variables should not be necessary anymore. However, I have difficulties to reproduce the old results with Singular's new implementation. The example from letterplace.pyx:

sage: F.<x,y,z> = FreeAlgebra(QQ, implementation='letterplace', degrees=[1,2,3])
sage: I = F * [x*y+z-y*x, x*y*z-x^6+y^3] * F
sage: I.groebner_basis(Infinity)
...
sage: F.degbound()
22

The maximum degree was 22, so the following Singular code should compute the same, as far as I understand, but it fails with an error:

LIB "freegb.lib";
ring r = 0,(x,y,z),wp(1,2,3);
def R = freeAlgebra(r, 22);
setring R;
ideal I = x*y+z-y*x, x*y*z-x^6+y^3;
ideal J = twostd(I);
//   ? degree bound of Letterplace ring is 22, but at least 23 is needed for this multiplication

Is this a bug in Singular? I think Singular should make sure not to go beyond the degree bound in this computation.

mwageringel commented 4 years ago
comment:55

I have posted this problem to the Singular forums.

mwageringel commented 4 years ago

Changed commit from 4bd32fe to 0c07123

mwageringel commented 4 years ago
comment:56

Viktor Levandovskyy has sent a response to the problem, which was very helpful. The ordering I was trying to use is not actually supported. Also, some of the orderings Sage was using in the letterplace computations were not actually supported and, apparently, were silently ignored by Singular. As a consequence, results obtained with the new Letterplace API (which now supports some of the orders) may be a bit different than before. In the case of non-standard degrees, a deglex ordering seems to give similar results as before.

Here is a tentative branch that makes Sage's letterplace functionality work with Singular 4.1.2p1. I followed Antonio's suggestion from comment:35 and his downstream patch about implementing a variable shifting method, and implemented a workaround to call Singular's twostd with a letterplace polynomial ring constructed via freeAlgebra. Of course, this should only be seen as a short-term solution, to maintain backward compatibility where possible.

There is one remaining problem for upgrading to Singular 4.1.2p1. The tests in src/sage/rings/polynomial/plural.pyx sometimes produce the segmentation fault reported in comment:22, which I do not know how to resolve. For me, the segfault usually occurs during make ptestlong, but not always when testing the file standalone.

Any help with this is appreciated.

New commits:

0ff739025993: upgrade to singular 4.1.2p1
71c871f25993: rename polynomial shift method to _cycle
0c0712325993: call new Singular API for letterplace algebras
mwageringel commented 4 years ago

Changed work issues from update letterplace algebras to segfault in plural.pyx

mwageringel commented 4 years ago

Changed branch from u/jdemeyer/ticket/25993 to u/gh-mwageringel/singular412p1

antonio-rojas commented 4 years ago
comment:57

Good news, the plural.pyx segfault seems fixed in 4.1.2.p3. There are a few additional test failures, but they look mostly harmless, caused by doc output changes and different generators for some ideals. There's also a small regression in the pc file https://www.singular.uni-kl.de:8005/trac/ticket/861

kiwifb commented 4 years ago
comment:58

fails its own testsuite on gentoo for stupid reason

make  check-TESTS
make[4]: Entering directory '/dev/shm/portage/sci-mathematics/singular-4.1.2_p3/work/singular-4.1.2/libpolys/polys'
make[5]: Entering directory '/dev/shm/portage/sci-mathematics/singular-4.1.2_p3/work/singular-4.1.2/libpolys/polys'
../../build-aux/test-driver: line 107: 707661 Segmentation fault      (core dumped) "$@" > $log_file 2>&1
FAIL: test
============================================================================
Testsuite summary for libpolys 4.1.2
============================================================================
# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See polys/test-suite.log

And when I look at the core dump

#0  0x00007f272947b23a in p_Add_q__FieldGeneral_LengthOne_OrdPomog () from /usr/libexec/singular/MOD/p_Procs_FieldGeneral.so
#1  0x00005583ef7743b8 in TestSum(ip_sring*, int) ()
#2  0x00005583ef775c03 in Test(ip_sring*) ()
#3  0x00005583ef77618f in test_Z13_t_GF() ()
#4  0x00005583ef77236f in main ()

the stupid thing tries to load a plugin from a previous install rather than the one it just compiled.

Tests pass on a machine with singular removed but it is annoying.

antonio-rojas commented 4 years ago
comment:59

4.1.2p4 has just been released with a fix for the pkgconfig issue

kiwifb commented 4 years ago
comment:60

My test suite issue is now https://github.com/Singular/Sources/issues/980 but if everything else can be worked out we should proceed. I'll add that flint detection is stinky and poorly thought out. It kinds of works but leads to messy warnings in the linker in some systems.

mwageringel commented 4 years ago

Changed commit from 0c07123 to 9dbc2ba

mwageringel commented 4 years ago
comment:61

I have updated the branch to fix the doctests for Singular 4.1.2p4. The changes are mostly harmless:

New commits:

3af226fMerge tag '9.1.beta4' into #25993
9dbc2ba25993: upgrade to singular 4.1.2p4
mwageringel commented 4 years ago

Changed branch from u/gh-mwageringel/singular412p1 to u/gh-mwageringel/singular412p4

mwageringel commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,4 +1,2 @@
-**Tarball 4.1.2p1**: https://service.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-1-2/singular-4.1.2p1.tar.gz
+**Tarball 4.1.2p4**: ftp://jim.mathematik.uni-kl.de/pub/Math/Singular/SOURCES/4-1-2/singular-4.1.2p4.tar.gz

-**Tarball 4.1.2p2**: ftp://jim.mathematik.uni-kl.de/pub/Math/Singular/SOURCES/4-1-2/singular-4.1.2p2.tar.gz
-
mwageringel commented 4 years ago

Changed work issues from segfault in plural.pyx to none

mwageringel commented 4 years ago

Work Issues: failing doctests in asymptotics_multivariate_generating_functions.py

mwageringel commented 4 years ago
comment:62

The only remaining failing doctests are in asymptotics_multivariate_generating_functions.py. It would be nice if the authors or anyone familiar with asymptotics could take a look.

Some of these are caused by different representatives being chosen modulo an ideal, which should be fine. However, the relative error does not get small anymore which suggests that there might be a bug on the Sage side.

File "src/sage/rings/asymptotic/asymptotics_multivariate_generating_functions.py", line 1576, in sage.rings.asymptotic.asymptotics_multivariate_generating_functions.FractionWithFactoredDenominator.?
Failed example:
    decomp = F.asymptotic_decomposition(alpha); decomp
Expected:
    (0, []) + (-3/2*r*(1/y + 1) - 1/2/y - 1/2, [(x*y + x + y - 1, 1)])
Got:
    (0, []) + (2*r*(1/x + 1) + 1/2/x - 1/2, [(x*y + x + y - 1, 1)])
**********************************************************************
File "src/sage/rings/asymptotic/asymptotics_multivariate_generating_functions.py", line 1585, in sage.rings.asymptotic.asymptotics_multivariate_generating_functions.FractionWithFactoredDenominator.?
Failed example:
    asy
Expected:
    (1/6000*(3600*sqrt(5)*sqrt(3)*sqrt(2)*sqrt(r)/sqrt(pi)
      + 463*sqrt(5)*sqrt(3)*sqrt(2)/(sqrt(pi)*sqrt(r)))*432^r,
     432,
     3/5*sqrt(5)*sqrt(3)*sqrt(2)*sqrt(r)/sqrt(pi)
      + 463/6000*sqrt(5)*sqrt(3)*sqrt(2)/(sqrt(pi)*sqrt(r)))
Got:
    (-1/6000*(3600*sqrt(5)*sqrt(3)*sqrt(2)*sqrt(r)/sqrt(pi) - 137*sqrt(5)*sqrt(3)*sqrt(2)/(sqrt(pi)*sqrt(r)))*432^r,
     432,
     -3/5*sqrt(5)*sqrt(3)*sqrt(2)*sqrt(r)/sqrt(pi) + 137/6000*sqrt(5)*sqrt(3)*sqrt(2)/(sqrt(pi)*sqrt(r)))
**********************************************************************
File "src/sage/rings/asymptotic/asymptotics_multivariate_generating_functions.py", line 1591, in sage.rings.asymptotic.asymptotics_multivariate_generating_functions.FractionWithFactoredDenominator.?
Failed example:
    F.relative_error(asy[0], alpha, [1, 2, 4, 8, 16], asy[1])  # abs tol 1e-10
Expected:
    [((4, 3), 2.083333333, [2.092576110], [-0.004436533009]),
     ((8, 6), 2.787374614, [2.790732875], [-0.001204811281]),
     ((16, 12), 3.826259447, [3.827462310], [-0.0003143703383]),
     ((32, 24), 5.328112821, [5.328540787], [-0.00008032230388]),
     ((64, 48), 7.475927885, [7.476079664], [-0.00002030232879])]
Got:
    [((4, 3), 2.083333333, [-1.783556749], [1.856107239]),
     ((8, 6), 2.787374614, [-2.572223188], [1.922812160]),
     ((16, 12), 3.826259447, [-3.672952629], [1.959932979]),
     ((32, 24), 5.328112821, [-5.219285944], [1.979574968]),
     ((64, 48), 7.475927885, [-7.398824824], [1.989686489])]
Tolerance exceeded in 10 of 25:
    2.092576110 vs -1.783556749, tolerance 4e0 > 1e-10
    -0.004436533009 vs 1.856107239, tolerance 2e0 > 1e-10
    2.790732875 vs -2.572223188, tolerance 6e0 > 1e-10
    -0.001204811281 vs 1.922812160, tolerance 2e0 > 1e-10
    3.827462310 vs -3.672952629, tolerance 8e0 > 1e-10
    -0.0003143703383 vs 1.959932979, tolerance 2e0 > 1e-10
    5.328540787 vs -5.219285944, tolerance 2e1 > 1e-10
    -0.00008032230388 vs 1.979574968, tolerance 2e0 > 1e-10
    7.476079664 vs -7.398824824, tolerance 2e1 > 1e-10
    -0.00002030232879 vs 1.989686489, tolerance 2e0 > 1e-10
**********************************************************************
File "src/sage/rings/asymptotic/asymptotics_multivariate_generating_functions.py", line 1609, in sage.rings.asymptotic.asymptotics_multivariate_generating_functions.FractionWithFactoredDenominator.?
Failed example:
    decomp = F.asymptotic_decomposition(alpha); decomp
Expected:
    (0, []) +
    (-16*r*(3/y - 4/z) - 16/y + 32/z,
     [(x + 2*y + z - 4, 1), (2*x + y + z - 4, 1)])
Got:
    (0, []) + (-16*r*(3/x - 2/z) - 16/x + 16/z, [(x + 2*y + z - 4, 1), (2*x + y + z - 4, 1)])
**********************************************************************
File "src/sage/rings/asymptotic/asymptotics_multivariate_generating_functions.py", line 1620, in sage.rings.asymptotic.asymptotics_multivariate_generating_functions.FractionWithFactoredDenominator.?
Failed example:
    asy # long time
Expected:
    (4/3*sqrt(3)*sqrt(r)/sqrt(pi) + 47/216*sqrt(3)/(sqrt(pi)*sqrt(r)),
     1, 4/3*sqrt(3)*sqrt(r)/sqrt(pi) + 47/216*sqrt(3)/(sqrt(pi)*sqrt(r)))
Got:
    (-4/3*sqrt(3)*sqrt(r)/sqrt(pi) - 47/216*sqrt(3)/(sqrt(pi)*sqrt(r)),
     1,
     -4/3*sqrt(3)*sqrt(r)/sqrt(pi) - 47/216*sqrt(3)/(sqrt(pi)*sqrt(r)))
**********************************************************************
File "src/sage/rings/asymptotic/asymptotics_multivariate_generating_functions.py", line 1623, in sage.rings.asymptotic.asymptotics_multivariate_generating_functions.FractionWithFactoredDenominator.?
Failed example:
    F.relative_error(asy[0], alpha, [1, 2, 4, 8], asy[1]) # long time
Expected:
    [((3, 3, 2), 0.9812164307, [1.515572606], [-0.54458543...]),
     ((6, 6, 4), 1.576181132, [1.992989399], [-0.26444185...]),
     ((12, 12, 8), 2.485286378, [2.712196351], [-0.091301338...]),
     ((24, 24, 16), 3.700576827, [3.760447895], [-0.016178847...])]
Got:
    [((3, 3, 2), 0.9812164307, [-1.515572606], [2.544585434]),
     ((6, 6, 4), 1.576181132, [-1.992989399], [2.264441858]),
     ((12, 12, 8), 2.485286378, [-2.712196351], [2.091301338]),
     ((24, 24, 16), 3.700576827, [-3.760447895], [2.016178847])]
**********************************************************************
antonio-rojas commented 4 years ago
comment:63

Looks like #29247 has broken things again. Getting many failures in sage/algebras/letterplace with this branch on top of 9.1.beta7

mwageringel commented 4 years ago
comment:64

Replying to @antonio-rojas:

Looks like #29247 has broken things again. Getting many failures in sage/algebras/letterplace with this branch on top of 9.1.beta7

This will be solved by #29311.

mwageringel commented 4 years ago

Dependencies: #29311

mkoeppe commented 4 years ago
comment:65

Related to this upgrade ticket: #29335, #29339, #27764

mkoeppe commented 4 years ago
comment:67

Is this ticket going to be ready for 9.1? Otherwise, we may need a ticket for 9.1 with some portability fixes (see #29415)

mwageringel commented 4 years ago
comment:68

Replying to @mkoeppe:

Is this ticket going to be ready for 9.1?

Probably not. There are still failing doctests related to asymptotics. Someone should look into that.

But even then, it might be better to merge this in the next release cycle in order to give it some time to get tested.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

5692f8b29465: fix sign error in cohomology_decomposition of asymptotics
2a09d15Merge branch #29465 into #25993
bdf81be25993: update doctests in asymptotics for Singular 4.1.2p4
78f439e25993: add upstream_url for singular
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 9dbc2ba to 78f439e

mwageringel commented 4 years ago

Author: Antonio Rojas, Markus Wageringel

mwageringel commented 4 years ago
comment:70

I found the problem in the asymptotics code. It is caused by an incorrect sign and is resolved by #29465, needing review.

Now this makes all the doctests pass with Singular 4.1.2p4, so this is ready for testing. The optional packages in particular still need to be tested.

Meanwhile, Singular 4.1.2p5 has been released and produces many segmentation faults all over the place, so I guess we stick to 4.1.2p4 for now.

mwageringel commented 4 years ago

Changed work issues from failing doctests in asymptotics_multivariate_generating_functions.py to none

mwageringel commented 4 years ago

Changed dependencies from #29311 to #29465

antonio-rojas commented 4 years ago
comment:71

The segfaults are fixed with https://github.com/Singular/Sources/commit/7886c15a361119b200bd15faef4bf5b620d783ad

4.1.3 is out with just one additional, harmless test failure

diff --git a/src/sage/libs/singular/function.pyx b/src/sage/libs/singular/function.pyx
index b649ab1e64..72f34fd67d 100644
--- a/src/sage/libs/singular/function.pyx
+++ b/src/sage/libs/singular/function.pyx
@@ -1308,7 +1308,7 @@ cdef class SingularFunction(SageObject):
             ...
             RuntimeError: error in Singular function call 'triangL':
             The input is no groebner basis.
-            leaving triang.lib::triangL
+            leaving triang.lib::triangL (0)

         Flush any stray output -- see :trac:`28622`::
mwageringel commented 4 years ago
comment:72

Thanks for the note. I have updated the branch.

New commits:

8722c0f25993: upgrade to singular 4.1.3
mwageringel commented 4 years ago

Changed branch from u/gh-mwageringel/singular412p4 to public/singular4-1-3

mwageringel commented 4 years ago

Changed commit from 78f439e to 8722c0f

mwageringel commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,2 +1,3 @@
-**Tarball 4.1.2p4**: ftp://jim.mathematik.uni-kl.de/pub/Math/Singular/SOURCES/4-1-2/singular-4.1.2p4.tar.gz
+**Tarball 4.1.3**: ftp://jim.mathematik.uni-kl.de/pub/Math/Singular/SOURCES/4-1-3/singular-4.1.3.tar.gz

+Use `make SAGE_SPKG="sage-spkg -o" singular-clean build` to automatically download and install.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

775f92325993: Merge tag '9.1.rc0' into #25993
5e496ab25993: remove patch "Fix building singular with -DNDEBUG"
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 8722c0f to 5e496ab

mwageringel commented 4 years ago
comment:74

I have merged 9.1.rc0 and removed the (backported) patch from #29438.

With this, I get the segmentation fault in plural.pyx from comment:22 again. I am not sure what to do with this. Maybe we could mark it as not tested and open a new ticket for it.