search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.3k stars 449 forks source link

Upgrade: Maxima 5.44.0 #30063

Closed pjbruin closed 4 years ago

pjbruin commented 4 years ago

Upstream tarball: see upstream_url in build/pkgs/maxima/checksums.ini.

Previous update: #26625 Update maxima to 5.42.2

Follow-up bug: #30389

Depends on #22191

CC: @dimpase @kiwifb @timokau @saraedum @slel @antonio-rojas @kcrisman @nbruin @paulmasson

Component: packages: standard

Keywords: upgrade, maxima

Author: Matthias Koeppe, Thierry Monteil

Branch: 21e2fea

Reviewer: Emmanuel Charpentier, Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/30063

mkoeppe commented 4 years ago

Dependencies: #22191

mkoeppe commented 4 years ago

Branch: u/mkoeppe/upgrade_maxima_to_5_44_0

mkoeppe commented 4 years ago

Commit: abb763e

mkoeppe commented 4 years ago
comment:3

Patches need to be removed or updated.

Last 10 new commits:

1d074e8doctest fixes
266d8c1backport ECL PR #210
12447bcreject old makeinfo
a3e0ecaadd upstream fix from MR 215
89b006badd the patch from upstream MR 214
0b77737add upstream MR 216 (to fix cygwin fork)
8ca1c0eMerge tag '9.2.beta2' into t/22191/public/packages/ecl20
f82c716Commit 75877dd8 from upstream
324869aMerge branch 'public/packages/ecl20' of git://trac.sagemath.org/sage into t/30063/upgrade_maxima_to_5_44_0
abb763ebuild/pkgs/maxima: Update to 5.44.0
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -1 +1,3 @@
-Upstream tarball can be downloaded via https://sourceforge.net/projects/maxima/files/Maxima-source/5.44.0-source/
+Upstream tarball: see tox.ini
+
+
mkoeppe commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,3 +1,5 @@
 Upstream tarball: see tox.ini

+Previous update: #26625 Update maxima to 5.42.2
+
slel commented 4 years ago

Description changed:

--- 
+++ 
@@ -1,5 +1,4 @@
-Upstream tarball: see tox.ini
-
+Upstream tarball: see `upstream_url` in `checksums.ini`.

 Previous update: #26625 Update maxima to 5.42.2
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

3db87adbuild/pkgs/maxima: Update to 5.44.0
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from abb763e to 3db87ad

mkoeppe commented 4 years ago
comment:8

Removing bugfix3629.patch, which is in 5.44.0

0001-taylor2-Avoid-blowing-the-stack-when-diff-expand-isn.patch does not apply. 5.44.0 has a different (simpler) change in this code.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

bcf3c98Remove old patches
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 3db87ad to bcf3c98

mkoeppe commented 4 years ago
comment:10

On macOS,

[maxima-5.44.0] /bin/bash /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/var/tmp/sage/build/maxima-5.44.0/src/missing makeinfo --html -c TEXI2HTML=1 --split=chapter --document-lang=en --output=. --css-include=../../../doc/info/manual.css --init-file  ../../../doc/info/texi2html.init  xmaxima.texi
[maxima-5.44.0] makeinfo: invalid option -- c
[maxima-5.44.0] Try `makeinfo --help' for more information.
mkoeppe commented 4 years ago
comment:11

This is /usr/bin/makeinfo

$ makeinfo --version
makeinfo (GNU texinfo) 4.8

Copyright (C) 2004 Free Software Foundation, Inc.
7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from bcf3c98 to 07aa2f9

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

07aa2f9build/pkgs/maxima/spkg-install.in: New workaround for makeinfo trouble
mkoeppe commented 4 years ago
comment:13

Builds OK now on macOS. Haven't tested anything else

mkoeppe commented 4 years ago
comment:14

Tests run at https://github.com/mkoeppe/sage/actions/runs/174422230

dimpase commented 4 years ago
comment:15

Replying to @sagetrac-git:

Branch pushed to git repo; I updated commit sha1. New commits:

07aa2f9build/pkgs/maxima/spkg-install.in: New workaround for makeinfo trouble

don't they have an option not to build docs?

mkoeppe commented 4 years ago
comment:16

Unfortunately they don't.

antonio-rojas commented 4 years ago
comment:17

These are the test failures I get downstream (using system maxima):

**********************************************************************
File "/usr/lib/python3.8/site-packages/sage/interfaces/maxima_abstract.py", line 225, in sage.interfaces.maxima_abstract.MaximaAbstract.example
Failed example:
    maxima.example('arrays')
Expected:
    a[n]:=n*a[n-1]
                                    a  := n a
                                     n       n - 1
    a[0]:1
    a[5]
                                          120
    a[n]:=n
    a[6]
                                           6
    a[4]
                                          24
                                         done
Got:
                                     n       n - 1
    a[0]:1
    a[5]
                                          120
    a[n]:=n
    a[6]
                                           6
    a[4]
                                          24
                                         done
    <BLANKLINE>
**********************************************************************
File "/usr/lib/python3.8/site-packages/sage/interfaces/maxima_abstract.py", line 283, in sage.interfaces.maxima_abstract.MaximaAbstract.completions
Failed example:
    sorted(maxima.completions('gc', verbose=False))
Expected:
    ['gcd', 'gcdex', 'gcfactor', 'gctime']
Got:
    ['gcd', 'gcd\\-impl', 'gcdex', 'gcfactor', 'gcfactor\\-impl', 'gctime']
**********************************************************************
File "/usr/lib/python3.8/site-packages/sage/symbolic/expression.pyx", line 10031, in sage.symbolic.expression.Expression.simplify_hypergeometric
Failed example:
    hypergeometric_M(1, 3, x).simplify_hypergeometric()
Expected:
    -2*(x - e^x + 1)/x^2
Got:
    2*e^x*gamma_incomplete_lower(2, x)/x^2
**********************************************************************
File "/usr/lib/python3.8/site-packages/sage/symbolic/integration/integral.py", line 864, in sage.symbolic.integration.integral.integrate
Failed example:
    a = integral(log(cot(x) - 1), x, 0, pi/4); a  # long time (about 6 s)
Expected:
    -1/4*pi*log(2) - 1/2*I*dilog(I + 1) + 1/2*I*dilog(-I + 1) + 1/2*I*dilog(1/2*I + 1/2) - 1/2*I*dilog(-1/2*I + 1/2)
Got:
    1/4*pi*(I*pi + log(2)) - 1/2*pi*log(2) - 1/2*I*dilog(I + 1) + 1/2*I*dilog(-I + 1) + 1/2*I*dilog(1/2*I + 1/2) - 1/2*I*dilog(-1/2*I + 1/2)
**********************************************************************
File "/usr/lib/python3.8/site-packages/sage/symbolic/integration/integral.py", line 866, in sage.symbolic.integration.integral.integrate
Failed example:
    abs(N(a - pi*log(2)/8)) < 1e-15  # long time
Expected:
    True
Got:
    False
**********************************************************************

The first one needs a readjustment of the number of removed lines in https://github.com/sagemath/sagetrac-mirror/blob/develop/src/sage/interfaces/maxima_abstract.py#n183

The ones in integration/integral.py are a known upstream issue since 5.43. The corresponding tests in maxima are disabled as known to fail, so maybe they can be tagged as known bug here.

The other two seem harmless.

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 07aa2f9 to 046634d

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

046634dsage.interfaces.maxima_abstract.MaximaAbstract._command_runner: Skip all comment lines, rather than counting lines to skip
mkoeppe commented 4 years ago
comment:19

Replying to @antonio-rojas:

The first one needs a readjustment of the number of removed lines in https://github.com/sagemath/sagetrac-mirror/blob/develop/src/sage/interfaces/maxima_abstract.py#n183

I have fixed the first one by a method that I hope to be more robust.

mkoeppe commented 4 years ago
comment:20

Replying to @antonio-rojas:

These are the test failures I get downstream (using system maxima):

**********************************************************************
File "/usr/lib/python3.8/site-packages/sage/interfaces/maxima_abstract.py", line 283, in sage.interfaces.maxima_abstract.MaximaAbstract.completions
Failed example:
    sorted(maxima.completions('gc', verbose=False))
Expected:
    ['gcd', 'gcdex', 'gcfactor', 'gctime']
Got:
    ['gcd', 'gcd\\-impl', 'gcdex', 'gcfactor', 'gcfactor\\-impl', 'gctime']
**********************************************************************

This is now https://sourceforge.net/p/maxima/bugs/3643/

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from 046634d to d1ad8d7

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

d1ad8d7sage.interfaces.maxima_abstract.MaximaAbstract.completions: Remove '-impl' functions
mkoeppe commented 4 years ago
comment:22

Replying to @antonio-rojas:

These are the test failures I get downstream (using system maxima):

**********************************************************************
File "/usr/lib/python3.8/site-packages/sage/symbolic/expression.pyx", line 10031, in sage.symbolic.expression.Expression.simplify_hypergeometric
Failed example:
    hypergeometric_M(1, 3, x).simplify_hypergeometric()
Expected:
    -2*(x - e^x + 1)/x^2
Got:
    2*e^x*gamma_incomplete_lower(2, x)/x^2

For this one I hope we can enroll the help of some symbolics experts. A related older ticket: #16697

dimpase commented 4 years ago
comment:23

Replying to @mkoeppe:

Replying to @antonio-rojas:

These are the test failures I get downstream (using system maxima):

**********************************************************************
File "/usr/lib/python3.8/site-packages/sage/symbolic/expression.pyx", line 10031, in sage.symbolic.expression.Expression.simplify_hypergeometric
Failed example:
    hypergeometric_M(1, 3, x).simplify_hypergeometric()
Expected:
    -2*(x - e^x + 1)/x^2
Got:
    2*e^x*gamma_incomplete_lower(2, x)/x^2

For this one I hope we can enroll the help of some symbolics experts. A related older ticket: #16697

M(1,3,x) = 1F1(1,3,x) - a.k.a Kummer function, whereas lower incomplete gamma is gamma(a,x)=xa/a 1F1(a,a+1,-x), a=2, i.e. the claim is that 1F1(1,3,x)=-exp(x) 1F1(2,3,-x). So this seems to be Kummer transform from https://en.wikipedia.org/wiki/Confluent_hypergeometric_function#Kummer's_transformation

but with a wrong sign (modulo typos...)

see also https://dlmf.nist.gov/13.6#E5

mkoeppe commented 4 years ago
comment:24

From the maxima 5.43 change log:

From the maxima 5.44 change log:

I note the following:

sage: hypergeometric_M(1, 3, 1).n()
1.43656365691809
sage: hypergeometric_M(1, 3, 1).simplify_hypergeometric()
2*e*gamma_incomplete_lower(2, 1)
sage: hypergeometric_M(1, 3, 1).simplify_hypergeometric().n()
TypeError: cannot evaluate symbolic expression numerically
kcrisman commented 4 years ago
comment:26

For the problem in comment:22 I recommend doing as Nils mentions in the recent sage-devel discussion around a Maxima infinite recursion error, and isolate which Maxima commands do this. I believe simplify_hypergeometric is a Maxima command wrapper, so it should be completely replicated in Maxima. By the way, in diagnosing this, a a few things to note.

sage: hypergeometric_M(1, 3, x)
hypergeometric_M(1, 3, x)
sage: maxima_calculus(hypergeometric_M(1,3,x))
kummer_m(1,3,_SAGE_VAR_x)

but

sage: B = hypergeometric_M(1, 3, x)
sage: B.generalized()
hypergeometric((1,), (3,), x)
sage: maxima_calculus(B.generalized())
hypergeometric([1],[3],_SAGE_VAR_x)

and also don't forget

sage: B.simplify_hypergeometric(algorithm='sage')
-2*(x - e^x + 1)/x^2

Unfortunately I don't have a chance to check out this update in order to diagnose the problem further. I do note that the hgfred uses maxima and not maxima_calculus, though perhaps it is imported as such earlier in the file.

edd8e884-f507-429a-b577-5d554626c0fe commented 4 years ago

Changed branch from u/mkoeppe/upgrade_maxima_to_5_44_0 to u/tmonteil/upgrade_maxima_to_5_44_0

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Changed commit from d1ad8d7 to 21e2fea

7ed8c4ca-6d56-4ae9-953a-41e42b4ed313 commented 4 years ago

Branch pushed to git repo; I updated commit sha1. New commits:

21e2fea#30063 : mark Maxima's integration regression as known bug.
edd8e884-f507-429a-b577-5d554626c0fe commented 4 years ago
comment:30

I was working on upgrading Maxima too (as i did not see that ticket), so while i am not a symbolics expert, i fixed the gamma_incomplete_lower issue: that function was called gamma_greek by Maxima before.

I also marked the intergration regression as known bug.

I think it is ready for review now.

edd8e884-f507-429a-b577-5d554626c0fe commented 4 years ago

Author: Matthias Koeppe, Thierry Monteil

7822f248-ba56-45f1-ab3d-4de7482bdf9f commented 4 years ago
comment:31

I'm trying to test this one. make started to getgcc 9.2.0 (whereas the system's gcc is at 10.1.0).

WTF ?

edd8e884-f507-429a-b577-5d554626c0fe commented 4 years ago
comment:32

Replying to @EmmanuelCharpentier:

I'm trying to test this one. make started to getgcc 9.2.0 (whereas the system's gcc is at 10.1.0).

WTF ?

This is probably because #29674 has been merged. Is it not related to the current ticket.

7822f248-ba56-45f1-ab3d-4de7482bdf9f commented 4 years ago
comment:33

Replying to @sagetrac-tmonteil:

Replying to @EmmanuelCharpentier:

I'm trying to test this one. make started to getgcc 9.2.0 (whereas the system's gcc is at 10.1.0).

WTF ?

This is probably because #29674 has been merged. Is it not related to the current ticket.

That was merged three months ago. I have rebuild and|or ipgraded my sage installation a few times since, and this never happened.

Furthermore, I tried reconfiguring with ./configure CC=gcc-9 CXX=g++-9 FC=gfortran-9 --enable-download-from-upstream-url ; configure now wants to install a bunch of system packages that were accepted before. Its answers :

configure: notice: the following SPKGs did not find equivalent system packages: cbc coxeter3 fflas_ffpack gfortran gsl iml openblas r suitesparse
checking for the package system in use... debian
configure: hint: installing the following system packages is recommended and may avoid building some of the above SPKGs from source:
configure:   $ sudo apt-get update 
  $ sudo apt-get install coinor-cbc coinor-libcbc-dev fflas-ffpack gfortran libgsl-dev libiml-dev libopenblas-dev r-base-dev r-cran-lattice libsuitesparse-dev
configure: After installation, re-run configure using:
configure:   $ ./config.status --recheck && ./config.status

Attempting a (simulated) installation tells me that all those packages are already at their latest versions... and the check fails again.

I'm afraid to be stuck...

7822f248-ba56-45f1-ab3d-4de7482bdf9f commented 4 years ago
comment:34

Replying to @EmmanuelCharpentier:

[ Snip... ]

I'm afraid to be stuck...

Re-reading my terminal I may have typed a small but devastating typo. I cleaned up and retried, and things seem to be OK for now.

//Deeply// sorry for the noise...

mkoeppe commented 4 years ago
comment:35

Replying to @sagetrac-tmonteil:

I was working on upgrading Maxima too (as i did not see that ticket), so while i am not a symbolics expert, i fixed the gamma_incomplete_lower issue: that function was called gamma_greek by Maxima before.

I also marked the intergration regression as known bug.

I think it is ready for review now.

Thanks for working on this!

Do we actually know whether this new Maxima version brings any improvements that are relevant for us -- given that there is this regression?

mkoeppe commented 4 years ago
comment:36

Replying to @mkoeppe:

Do we actually know whether this new Maxima version brings any improvements that are relevant for us -- given that there is this regression?

OK, I see, #28538

mkoeppe commented 4 years ago
comment:37

Dima had a concern about a possible sign error in comment 23 above - could someone double check?

slel commented 4 years ago
comment:38

Tickets with this ticket as a dependency:

7822f248-ba56-45f1-ab3d-4de7482bdf9f commented 4 years ago
comment:39

Okay. ptestlong gives me the same (unrelated) failures as what I have already reported since 9.2.beta3 (IIRC).

Concerning comment 23 :

sage: hypergeometric_M(1, 3, x).simplify_hypergeometric()                       
-2*((x + 1)*e^(-x) - 1)*e^x/x^2
sage: hypergeometric_M(1, 3, x).simplify_hypergeometric().factor()              
-2*(x - e^x + 1)/x^2

which seems kosher :

sage: mathematica.FullSimplify(hypergeometric_M(1, 3, x)).sage().factor()       
-2*(x - e^x + 1)/x^2

==> (tentative) positive_review.

As always, cross-checks on various platforms are more than useful.

[ Only incidentally related... ] : I //sternuously// object to have to retain obsolete versions of the compilers in order to be able to compile without being told so. This problem, which appears only in some platforms, should be documented in the README.md ; furthermore, ./configure should :

Where should this rant be usefully posted ? I am aware of too much tickets related to system configuration and options, and am a bit lost...

seblabbe commented 4 years ago
comment:40

Reviewer name is missing, I am changing to needs work before the release manager needs to do it.

mkoeppe commented 4 years ago
comment:41

Also, this ticket should undergo portability testing before it should be set to "positive review".

mkoeppe commented 4 years ago
comment:42

Replying to @EmmanuelCharpentier:

./configure should :

  • suggest the installation of the relevant packages for older versions (recompiling the gcc smalah takes a geological age or two...), and
  • suggest reconfiguring with the relevant arguments.

Where should this ... be usefully posted ?

Try #29586 (Improve configure's recommendations). It would be helpful to include the messages that configure prints on your system, and to explain what is missing.

7822f248-ba56-45f1-ab3d-4de7482bdf9f commented 4 years ago

Reviewer: Emmanuel Charpentier

7822f248-ba56-45f1-ab3d-4de7482bdf9f commented 4 years ago
comment:44

Adding my name (as I should have initially...), but keeping it at needs_review.