search

sagemath / sage

Main repository of SageMath. Now open for Issues and Pull Requests.
https://www.sagemath.org
Other
1.09k stars 394 forks source link

Symbolic Ring to Maxima via EclObject #7377

Closed nbruin closed 13 years ago

nbruin commented 14 years ago

With maxima-as-an-ecl-library and ecl accessible as a library, we can start interfacing with maxima via a binary library interface. This should be more efficient and more robust, because expressions can be transmitted in a much richer format than text and parsing does not have to recognise error messages and questions (since communication does not go via STDIN/STDOUT anymore).

Applies cleanly to 4.7.alpha3.

Depends on: #10766, #10773, #10743. #10818 desirable but not essential.

Apply:

CC: @burcin @saliola @jpflori

Component: symbolics

Author: Robert Bradshaw, Nils Bruin, Jean-Pierre Flori, Karl-Dieter Crisman

Reviewer: Jean-Pierre Flori, François Bissey, Karl-Dieter Crisman, Nils Bruin

Merged: sage-4.7.1.alpha0

Issue created by migration from https://trac.sagemath.org/ticket/7377

nbruin commented 14 years ago

Attachment: sagemax.py.gz

example package for how to accomplish said interface

nbruin commented 14 years ago

I think the following email might be of interest to other people wanting to hack maxima:

Since there is already a decent string parser for taking Sage stuff to Maxima and (especially) getting back Sage stuff from Maxima, would there be a way to modify the above - or ecl_eval - to give back the Maxima expression directly? Perhaps this is already in the lib/ecl.pyx file, though I didn't see it on the first reading.

lib/ecl.pyx has nothing to do with maxima-specific stuff. That is all in sagemax.py. It is actually a little bit important to keep that distinction. Maxima is not the only lisp program we might want to interface with.

There is max_read(S) = cadadr(EclObject("#$%s"%S)), which reads the string S as a maxima expression and returns its parsed result.

So meval(max_read(S)) would do the trick. I do not know what routines maxima has to convert expressions to a string. If I wanted to know, I would find it out by looking at something like

max_read("maxima-command-to-print-to-string(x)")

and see what token the maxima reader puts in. I imagine you get something like

<ECL: ((MAX-PRINT-STRING) x)>

back, So do:

maxprint:=EclObject("MAX-PRINT-STRING") #do this once globally
meval(EclObject([[maxprint], maxexpression])).python()

Indeed:

sage: max_read("string(x)")
<ECL: (($STRING) $X)>
sage: meval(max_read("string(x)"))
<ECL: "x">
sage: meval(max_read("string(x)")).python()
'"x"'
sage: meval(max_read("string(x)")).python()[1:-1]
'x'

So

sage: maxprint=EclObject("$STRING")
sage: def max_to_string(s):
....:      return meval(EclObject([[maxprint],s])).python()[1:-1]

would give you a string representation. Be aware that #$...$ suffers from performance loss, so if you repeatedly use it, your timings will go south quickly. See Dodier's comments on this a while ago. Apparently you could patch Maxima to lose the performance-losing routine (at the expense of losing the wonderful linear-search history the maxima parser offers now).

Sage command -> calls Maxima command -> Maxima command evaluated, but in the ECL library version, not the pexpect version -> Maxima output (a string) converted back using the stuff in Sage

Yes, that would be an approach virtually orthogonal to the one taken in sagemax.py.

> sage: str(ecl_eval("#$load(to_poly_solver)$"))
> append: argument must be a non-atomic expression; found false
> -- an error.  To debug this try debugmode(true);

To illustrate the inner workings:

sage: from sagemax import *
sage: e=max_read('load("to_poly_solver")')
sage: e
<ECL: (($LOAD) "to_poly_solver")>
sage: meval(e)
append: argument must be a non-atomic expression; found false
 -- an error.  To debug this try debugmode(true);

sage: meval(max_read("append(a,b)"))
append: argument must be a non-atomic expression; found a
 -- an error.  To debug this try debugmode(true);

So my guess is that "load" is trying to append to an improperly initialized maxima list. I guess that means that in the current setting, our maxima environment hasn't properly been set up yet. That is no surprise. It is actually surprising that so much of maxima works without any initialization at all.

robertwb commented 14 years ago

Attachment: 7377-abstract-maxima.patch.gz

nbruin commented 14 years ago
comment:3

Finally success! Recipe for getting calculus use maxima in ecl library mode do the following

nbruin commented 14 years ago
comment:4

Updated maxlib.patch. Changes made:

Doctesting calculus/calculus.py leads to 10 failures. All have to do with either forget() acting up or the fact that "maxima asks a question" gets reported differently by the two interfaces.

(the uploaded code has some diagnostic output in maxima_eval to try to debug the "forget" issues. The "ask" errors should just consist of updating the doctest.

Go at it!

nbruin commented 14 years ago

Maxima via library; used by calculus

nbruin commented 14 years ago
comment:5

Attachment: maxlib.patch.gz

Another update:

Doctests failures on calculus.py are only 5 and are restricted to whitespace and differently formatted error messages (in other words, for real use, the two interfaces shouldn't be distinguishable anymore).

kcrisman commented 14 years ago
comment:7

Very sadly I do not have time right now to look at this properly, but if it behaves as advertised, the layout is very understandable and I think might even improve the previous interface in terms of future customization.

Do you think you can post some sample timings, just for information? After all, one (not the only) reason for this would be to significantly speed up repeated use of Maxima for certain applications.

Thanks to nbruin and robertwb for working so hard on this!

nbruin commented 14 years ago

Patch to get faster calculus routines

nbruin commented 14 years ago
comment:8

Attachment: fastcalculus.patch.gz

The latest patch must be applied after abstract-maxima and maxlib. It adds routines sr_to_max and max_to_sr that try to translate between maxima's internal datastructure and SR while avoiding string transformations as much as possible. It falls back to the old interface to establish many of its symbol and operator translations, but takes note of them and uses a dictionary directly the next time around. It also avoids the overhead of binding expressions to maxima variables.

to go back and forth, maxima_lib.MaximaElement() as a new function {ecl()} which passes back the underlying EclObject. Conversely, maxima_lib.maxima accepts EclObjects and returns a corresponding expect interface object.

This patch also rewrite calculus integral, limit and sum to directly pass back and forth to maxima. There is a lot of overhead in SR itself, so speed gains are not that big. New times are:

sage: timeit("integral(cos(x),x)")
625 loops, best of 3: 1.1 ms per loop
sage: timeit("integral(cos(x^2),x)")
5 loops, best of 3: 31.2 ms per loop

where old were

sage: timeit("integral(cos(x),x)")
25 loops, best of 3: 8.08 ms per loop
sage: timeit("integral(cos(x^2),x)")
5 loops, best of 3: 37 ms per loop

It is easy to swap the old and new interfaces: simply comment/uncomment the appropriate lines in calculus.py to define the appropriate calculus.maxima

There are 7 doctest failures in calculus.py. All of them are whitespace, errors and changed precision in floats: the binary interface passes doubles directly, so there are no digits rounded. This affects some doctests. In other words, as far is calculus.py is concerned the two interfaces are functionally equivalent.

kiwifb commented 13 years ago
comment:11

Does this need any rebasing? I am willing to give it a spin in sage-on-gentoo to see if there are any obvious problems on linux.

burcin commented 13 years ago
comment:12

Replying to @kiwifb:

Does this need any rebasing?

Unfortunately, it does. Since Robert split off the Maxima pexpect interface, there were a few minor changes to sage/interfaces/maxima.py. His patch needs to be rebased.

AFAIR, the only reason that held this ticket back was a mix-up in the updates to the ecl/maxima packages. The version that properly built maxima as a library got lost since there were several new packages at the same time. I don't know if that problem is fixed in the latest release.

bac7d3ea-3f1b-4826-8464-f0b53d5e12d2 commented 13 years ago
comment:13

Note that there's a ticket to update both Maxima and ECL open just now. See #10187

At this point in time, it does not have positive review, but I think it is pretty close to getting it. The packages work together, but there's a doctest failure which needs to be resolved. As far as I can see, its just that Maxima is outputing results in a different form (yet again).

I would suggest you use the packages at #10187 and make this dependent on that, which I think will be merged in the next alpha.

jpflori commented 13 years ago
comment:14

I tried to update the patches so that they apply on Sage 4.6.1 with Maxima 5.22.1, some things moved since the last patches were produced, but there was nothing too terrible. I'll up them right now.

However, there is now a problem when loading Maxima as a library. Maybe some changes in the spkg (in init-cl.lisp where set-pathnames is defined ?) are responsible for this. Without that line, there is a problem later.

/home/jp/boulot/sage/sage-current/local/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py in <module>()
    487 ecl_eval("(defvar *MAXIMA-LANG-SUBDIR* NIL)")
    488 ecl_eval("(set-locale-subdir)")
--> 489 ecl_eval("(set-pathnames)")
    490 ecl_eval("(defun add-lineinfo (x) x)")
    491 ecl_eval("""(defun retrieve (msg flag &aux (print? nil))(error (concatenate 'string "Maxima asks:" (meval (list '($string) msg)))))""")

/home/jp/boulot/sage/sage-current/local/lib/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_eval (sage/libs/ecl.c:5616)()

/home/jp/boulot/sage/sage-current/local/lib/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_eval (sage/libs/ecl.c:5570)()

/home/jp/boulot/sage/sage-current/local/lib/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_safe_eval (sage/libs/ecl.c:2403)()

RuntimeError: ECL says: In function PATHNAME, the value of the only argument is
  NIL
which is not of the expected type (OR FILE-STREAM STRING PATHNAME)
Error importing ipy_profile_sage - perhaps you should run %upgrade?
WARNING: Loading of ipy_profile_sage failed.

I don't know anything about Lisp, ECL and Maxima, so I was not able to fix that in the little time I tried.

As far as the patches as concerned, I feel that more work is needed. Deleting duplicate examples and methods, moving the library interface to libs directory, not sure where maxima abstract should go (if it is kept, should something be done like MaximaLib/Maxima as for Pari/GP ? no idea if those two "interfaces" share something...), getting rid of the inheritance on Expect in Maxima_lib and Maxima_abstract.

I'm ready to have a look at all of this, if someone helps me launching Maxima as library with latest versions of everything.

jpflori commented 13 years ago

Attachment: trac_7377-abstract-maxima-rebased.patch.gz

Rebased on Sage 4.6.1

jpflori commented 13 years ago

Rebased on Sage 4.6.1

kiwifb commented 13 years ago
comment:15

Attachment: trac_7377-maximalib-rebased.patch.gz

What happens if you just remove that line? I just noticed a typo - marking makes you terrible at that kind of things :) "We begin here by initializing maxima in library more" ^

jpflori commented 13 years ago
comment:16

It fails later, here is the piece of code involved:

/home/jp/boulot/sage/sage-current/local/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py in __init__(self, script_subdirectory, logfile, server, init_code)
    586         ecl_eval("(setf *standard-output* *dev-null*)")
    587         for l in init_code:
--> 588             ecl_eval("#$%s$"%l)
    589         ecl_eval("(setf *standard-output* original-standard-output)")
    590 

/home/jp/boulot/sage/sage-current/local/lib/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_eval (sage/libs/ecl.c:5616)()

/home/jp/boulot/sage/sage-current/local/lib/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_eval (sage/libs/ecl.c:5570)()

/home/jp/boulot/sage/sage-current/local/lib/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_safe_eval (sage/libs/ecl.c:2403)()

RuntimeError: ECL says: THROW: The catch MACSYMA-QUIT is undefined.
Error importing ipy_profile_sage - perhaps you should run %upgrade?
WARNING: Loading of ipy_profile_sage failed.

Tried to google that, but couldn't find anything meaningful to me.

kiwifb commented 13 years ago
comment:17

me neither :(

jpflori commented 13 years ago
comment:18

I don't know what happened but I rebuilt ecl and maxima while trying to add debug information and everything is fine now. I surely did something wrong at some point...

There are some problems left with what I uploaded earlier in sage.symbolic.integration.external I'll upload fixed version of the patches soon.

jpflori commented 13 years ago

Attachment: trac_7377-fastcalculus-rebased.patch.gz

Rebased on Sage 4.6.1, fixed changes in sage.symbolic.integration.external

kiwifb commented 13 years ago
comment:19

Ok I will give it a spin in sage-on-gentoo.

kiwifb commented 13 years ago
comment:20

I tried the patch against 4.6.2.alpha2 and it builds but it doesn't run properly here:

/usr/lib64/python2.6/site-packages/sage/calculus/calculus.py in <module>()
    375         definite_integral
    376 import sage.symbolic.pynac
--> 377 import sage.interfaces.maxima_lib
    378 
    379 """

/usr/lib64/python2.6/site-packages/sage/interfaces/maxima_lib.py in <module>()
    482 from sage.libs.ecl import *
    483 ecl_eval("(setf *load-verbose* NIL)")
--> 484 ecl_eval("(require 'maxima)")
    485 ecl_eval("(in-package :maxima)")
    486 ecl_eval("(setq $nolabels t))")

/usr/lib64/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_eval (sage/libs/ecl.c:5617)()
    992 
    993 
--> 994 
    995 
    996 

/usr/lib64/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_eval (sage/libs/ecl.c:5571)()
   1007 
   1008 
-> 1009 
   1010 
   1011 

/usr/lib64/python2.6/site-packages/sage/libs/ecl.so in sage.libs.ecl.ecl_safe_eval (sage/libs/ecl.c:2404)()
    184 
    185 
--> 186 
    187 
    188 

RuntimeError: ECL says: Module error: Don't know how to REQUIRE MAXIMA.
Error importing ipy_profile_sage - perhaps you should run %upgrade?
WARNING: Loading of ipy_profile_sage failed.

I am currently also running ecl-11.1.1 which is a source of trouble but I don't think it is responsible here.

nbruin commented 13 years ago
comment:21

RuntimeError: ECL says: Module error: Don't know how to REQUIRE MAXIMA. Error importing ipy_profile_sage - perhaps you should run %upgrade? WARNING: Loading of ipy_profile_sage failed.

My first guess is that maxima.fas is not available to your newly built ecl-11.1.1.

Note that in 4.6.1, building maxima produces this library:

$SAGE_ROOT/local/lib/ecl-10.4.1/maxima.fas

so once you upgrade ecl, you need to rebuild maxima as well. Did you do that?

kiwifb commented 13 years ago
comment:22

I downgraded to ecl-10.4.1 and rebuilt maxima and sage but I don't seem to have maxima.fas, I have a number of other fas but not maxima.fas. It was done on sage-on-gentoo, so it is possible that this particular file wasn't installed for a reason or another. I will check what the maxima spkg do.

kiwifb commented 13 years ago
comment:23

OK pity it is not build in maxima by default, works for me now. Not sure how i will sell that one to the maintainer of maxima in Gentoo. Even if I have a good relationship with him.

jpflori commented 13 years ago
comment:24

I just upgraded to 4.6.2.aplha2 (hoping I won't have to rebuild from scratch for next upgrade...) and everything went fine, it gives me better timings than with the Pexpect interface. Before patches:

Loading Sage library. Current Mercurial branch is: jp
sage: timeit("integral(cos(x^2),x)")
5 loops, best of 3: 96.6 ms per loop
sage: 
Exiting Sage (CPU time 0m1.02s, Wall time 0m8.17s).
Exiting spawned Maxima process.

After:

Loading Sage library. Current Mercurial branch is: jp
sage: timeit("integral(cos(x^2),x)")
5 loops, best of 3: 62.8 ms per loop
sage: 
Exiting Sage (CPU time 0m1.56s, Wall time 3m5.27s).

However those are much more than what was posted above.

Glad to hear it also ran on Sage on Gentoo.

I think some refactoring still has to be done for the new classes. The abstract class is a good idea to transparently switch interface but I think it would be cleaner if it did not inherit from Expect class (and so "Maxima as a lib" does not too). I'll try to work on this this weekend.

It should be also be decided, which interface is used for what, especially since we can only have one "Maxima as a lib" running. Should that Maxima as lib be used for calculus and be given a direct access (maybe as "maximalib" variable as "maxima" gives access to a copy of Maxima in Sage, even if currently the "maxima" variable points to a different instance of Maxima than the one used by calculus).

nbruin commented 13 years ago
comment:25

Replying to @jpflori:

Hi Jean-Pierre,

It's great you're taking on this project! It has the potential of being a lot of fun. I won't be working on it, so I'm very happy to see someone else interested in taking it on. I'll give you some background on the points you raise, so that you can take more informed design decisions.

I think some refactoring still has to be done for the new classes. The abstract class is a good idea to transparently switch interface but I think it would be cleaner if it did not inherit from Expect class (and so "Maxima as a lib" does not too). I'll try to work on this this weekend.

The only reason to organize the maxima interfaces as they are now is because Robert remarked that whatever was needed to communicate with maxlib at a string-level was going to have a lot of overlap with the present maxima interface. Hence, he abstracted out the common stuff and I modified the concrete maxlib interface to talk to ecllib instead of a separate process. This was experimental and at the time I had no idea how much was necessary, so I changed as little as possible: The original interface inherited from expect, so the new one did too, just overriding whatever methods needed overriding. So yes, there will be a lot of lint and extraneous stuff in there. What you're looking at is a first hack.

It should be also be decided, which interface is used for what, especially since we can only have one "Maxima as a lib" running. Should that Maxima as lib be used for calculus and be given a direct access (maybe as "maximalib" variable as "maxima" gives access to a copy of Maxima in Sage, even if currently the "maxima" variable points to a different instance of Maxima than the one used by calculus).

The largest benefit is to be gained when the communication can avoid string parsing altogether. (ecllib at the moment converts Integers to ecl bignums via strings, which is silly: They are both implemented as gmp/mpir integers! The only reason is that I never succeeded in importing the gmp/mpir headers in a usable way in ecl.pyx, to copy over the integer. ECL largely avoids gmp's registered memory manager, so some care must be taken to transfer the integer to properly allocated memory.) The routines sr_to_max and max_to_sr (in the fastcalculus patch) implement this strategy. If you want to have this benefit, the maxlib instance has to be the "calculus" instance. Note that the default "maxima" instance is already distinct from the "calculus" instance:

sage: integrate(cos(x),x);
sage: maxima(x);
sage: 
Exiting Sage (CPU time 0m0.12s, Wall time 0m16.73s).
Exiting spawned Maxima process.
Exiting spawned Maxima process.

(note the two maxima spawns)

Once the design has stabilized, it would be possible to make max_to_sr and sr_to_max more intelligent. For instance, SR variables would not need to be translated to maxima identifiers with the same name. This currently leads to problems, because this conversion collapses namespaces:

sage: cosvar=SR.var("cos")
sage: cos(cosvar)
cos(cos)
sage: integrate(cos(cosvar),cosvar)
integrate(cos(cos), cos)
sage: cosvar=SR.var("sage_cos")
sage: integrate(cos(cosvar),cosvar)
sin(sage_cos)

as you see, the print name of the SR.var matters!

Let me know if you have further questions. I might be able to help with information.

kcrisman commented 13 years ago
comment:26

Thanks so much, Nils and JP, for resurrecting this. JP, I stand ready to test anything you feel is ready to look at. If you know little about ECL, I know even less, so I can't do more than that, but this would be a great start to improving some of Sage's interface. It's true that the timing on average doesn't improve so greatly (yet), but the point is that not having to start Maxima up when you want to do an integral means more immediate gratification for the user. (Assuming immediate gratification is a good thing.)

kiwifb commented 13 years ago
comment:27

Ok so it runs but in its present state it breaks quite a lot of doctests for me. Unless you can point out something else I would need to do. example 1

sage -t  -force_lib "devel/sage/sage/symbolic/function_factory.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/symbolic/function_factory.py", line 174:
    sage: g
Expected:
    b*D[0](cr)(a)
Got:
    b*del(a)
**********************************************************************
File "/usr/share/sage/devel/sage/sage/symbolic/function_factory.py", line 184:
    sage: g.substitute_function(cr, cos)
Expected:
    -b*sin(a)
Got:
    b*del(a)
**********************************************************************
File "/usr/share/sage/devel/sage/sage/symbolic/function_factory.py", line 187:
    sage: g.substitute_function(cr, (sin(x) + cos(x)).function(x))
Expected:
    -(sin(a) - cos(a))*b
Got:
    b*del(a)
**********************************************************************
1 items had failures:
   3 of  58 in __main__.example_6

example 2

sage -t -force_lib "devel/sage/sage/interfaces/maxima.py"   
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima.py", line 426:
    sage: t.limit(Ax=0, dir='+')
Exception raised:
    Traceback (most recent call last):
      File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/usr/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_0[93]>", line 1, in <module>
        t.limit(Ax=Integer(0), dir='+')###line 426:
    sage: t.limit(Ax=0, dir='+')
      File "expression.pyx", line 8202, in sage.symbolic.expression.Expression.limit (sage/symbolic/expression.cpp:31252)
      File "/usr/lib/python2.6/site-packages/sage/calculus/calculus.py", line 1122, in limit
        return l.sage()
      File "element.pyx", line 327, in sage.structure.element.Element.__getattr__ (sage/structure/element.c:2715)
      File "parent.pyx", line 277, in sage.structure.parent.getattr_from_other_class (sage/structure/parent.c:2841)
      File "parent.pyx", line 175, in sage.structure.parent.raise_attribute_error (sage/structure/parent.c:2636)
    AttributeError: 'sage.rings.integer.Integer' object has no attribute 'sage'
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima.py", line 894:
    sage: f(3.2)
Expected:
    -.05837414342758009
Got:
    -.058374143427580086
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima.py", line 1006:
    sage: maxima.version()
Expected:
    '5.22.1'
Got:
    '5.23.2'
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima.py", line 1087:
    sage: maxima_version()
Expected:
    '5.22.1'
Got:
    '5.23.2'
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima.py", line 608:
    sage: integrate(1/(x^3 *(a+b*x)^(1/3)),x)
Expected:
    Traceback (most recent call last):
    ...
    TypeError: Computation failed since Maxima requested additional constraints (try the command 'assume(a>0)' before integral or limit evaluation, for example):
    Is  a  positive or negative?
Got:
    Traceback (most recent call last):
      File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/usr/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_6[3]>", line 1, in <module>
        integrate(Integer(1)/(x**Integer(3) *(a+b*x)**(Integer(1)/Integer(3))),x)###line 608:
    sage: integrate(1/(x^3 *(a+b*x)^(1/3)),x)
      File "/usr/lib/python2.6/site-packages/sage/misc/functional.py", line 718, in integral
        return x.integral(*args, **kwds)
      File "expression.pyx", line 8153, in sage.symbolic.expression.Expression.integral (sage/symbolic/expression.cpp:30871)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 601, in integrate
        return indefinite_integral(expression, v)
      File "function.pyx", line 419, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:4486)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 85, in _eval_
        res = integrator(f, x)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/external.py", line 19, in maxima_integrator
        result = maxima.sr_integral(expression,v)
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 983, in sr_integral
        raise error
    RuntimeError: ECL says: Maxima asks:?mtext("Is  ",a,"  positive or negative?")
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima.py", line 618:
    sage: integral(x^n,x)
Expected:
    Traceback (most recent call last):
    ...
    TypeError: Computation failed since Maxima requested additional constraints (try the command 'assume(n+1>0)' before integral or limit evaluation, for example):
    Is  n+1  zero or nonzero?
Got:
    Traceback (most recent call last):
      File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/usr/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_6[7]>", line 1, in <module>
        integral(x**n,x)###line 618:
    sage: integral(x^n,x)
      File "/usr/lib/python2.6/site-packages/sage/misc/functional.py", line 718, in integral
        return x.integral(*args, **kwds)
      File "expression.pyx", line 8153, in sage.symbolic.expression.Expression.integral (sage/symbolic/expression.cpp:30871)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 601, in integrate
        return indefinite_integral(expression, v)
      File "function.pyx", line 419, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:4486)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 85, in _eval_
        res = integrator(f, x)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/external.py", line 19, in maxima_integrator
        result = maxima.sr_integral(expression,v)
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 983, in sr_integral
        raise error
    RuntimeError: ECL says: Maxima asks:?mtext("Is  ",n+1,"  zero or nonzero?")
**********************************************************************
5 items had failures:
   1 of  95 in __main__.example_0
   1 of  16 in __main__.example_13
   1 of   3 in __main__.example_17
   1 of   4 in __main__.example_21
   2 of  11 in __main__.example_6
***Test Failed*** 6 failures.
For whitespace errors, see the file /home/francois/.sage/tmp/.doctest_maxima.py

and there are more. Tests are still running on this slow machine.

kcrisman commented 13 years ago
comment:28
sage -t  -force_lib "devel/sage/sage/symbolic/function_factory.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/symbolic/function_factory.py", line 174:
    sage: g
Expected:
    b*D[0](cr)(a)
Got:
    b*del(a)
**********************************************************************
File "/usr/share/sage/devel/sage/sage/symbolic/function_factory.py", line 184:
    sage: g.substitute_function(cr, cos)
Expected:
    -b*sin(a)
Got:
    b*del(a)
**********************************************************************
File "/usr/share/sage/devel/sage/sage/symbolic/function_factory.py", line 187:
    sage: g.substitute_function(cr, (sin(x) + cos(x)).function(x))
Expected:
    -(sin(a) - cos(a))*b
Got:
    b*del(a)
**********************************************************************

Hmm, that is very interesting. I am not quite sure what this del thing is, but I remember seeing something about it on the Maxima list...

File "/usr/share/sage/devel/sage/sage/interfaces/maxima.py", line 426: sage: t.limit(Ax=0, dir='+') return l.sage() File "element.pyx", line 327, in sage.structure.element.Element.getattr (sage/structure/element.c:2715) File "parent.pyx", line 277, in sage.structure.parent.getattr_from_other_class (sage/structure/parent.c:2841) File "parent.pyx", line 175, in sage.structure.parent.raise_attribute_error (sage/structure/parent.c:2636) AttributeError: 'sage.rings.integer.Integer' object has no attribute 'sage'

This must be us using Sage Integers somehow at the library level where before they were coming back from Maxima?

File "/usr/share/sage/devel/sage/sage/interfaces/maxima.py", line 608: sage: integrate(1/(x^3 (a+bx)^(1/3)),x) Expected: Traceback (most recent call last): ... TypeError: Computation failed since Maxima requested additional constraints (try the command 'assume(a>0)' before integral or limit evaluation, for example): Is a positive or negative?

Notice we expect a TypeError.

RuntimeError: ECL says: Maxima asks:?mtext("Is  ",a,"  positive or negative?")

Now we just get a different error. I assume that the new code interface has something about ECL asking. We would want to make sure that this still is turned into the helpful type of message we have here, but that should be easy (if tedious) to make work.

Thank you for beginning this testing!

kcrisman commented 13 years ago

Reviewer: Jean-Pierre Flori, Francois Bissey, Karl-Dieter Crisan

kcrisman commented 13 years ago

Changed author from Nils Bruin to Nils Bruin, Jean-Pierre Flori

kcrisman commented 13 years ago

Changed reviewer from Jean-Pierre Flori, Francois Bissey, Karl-Dieter Crisan to Jean-Pierre Flori, Francois Bissey, Karl-Dieter Crisman

kcrisman commented 13 years ago
comment:29

I don't have the cedille, pardon me :(

kiwifb commented 13 years ago
comment:30

I have a cedilla thanks to the international setting on my desktop.

More seriously, it should be pointed out that this is done with ecl-11.1.1 and it may have an influence. I have a faster machine at work and I'll test ecl-10.4.1 on Monday.

kiwifb commented 13 years ago

Changed reviewer from Jean-Pierre Flori, Francois Bissey, Karl-Dieter Crisman to Jean-Pierre Flori, François Bissey, Karl-Dieter Crisman

kiwifb commented 13 years ago
comment:31

Ok so I have a lot of failures that I attribute to maxima-lib, I am making a list for quick reference. I have another sage-on-gentoo report on there being a lot failure with it and this time with ecl-10.4.1.

sage -t -force_lib "devel/sage/sage/functions/special.py" # Time out

sage -t -force_lib "devel/sage/sage/functions/piecewise.py"

sage -t -force_lib "devel/sage/sage/functions/min_max.py"

sage -t -force_lib "devel/sage/sage/functions/orthogonal_polys.py"

sage -t -force_lib "devel/sage/sage/interfaces/maxima_lib.py"

sage -t -force_lib "devel/sage/sage/interfaces/maxima_abstract.py"

sage -t -force_lib "devel/sage/sage/interfaces/maxima.py"

sage -t -force_lib "devel/sage/sage/symbolic/function_factory.py"

sage -t -force_lib "devel/sage/sage/symbolic/integration/integral.py" # Time out

sage -t -force_lib "devel/sage/sage/symbolic/integration/external.py"

sage -t -force_lib "devel/sage/sage/symbolic/relation.py" # Time out

sage -t -force_lib "devel/sage/sage/symbolic/assumptions.py" # Time out

sage -t -force_lib "devel/sage/sage/symbolic/power_helper.pyx"

sage -t -force_lib "devel/sage/sage/symbolic/expression_conversions.py"

sage -t -force_lib "devel/sage/sage/symbolic/ring.pyx"

sage -t -force_lib "devel/sage/sage/symbolic/expression.pyx" # Time out

sage -t -force_lib "devel/sage/sage/rings/number_field/number_field_element_quadratic.pyx"

sage -t -force_lib "devel/sage/sage/rings/number_field/number_field_element.pyx"

sage -t -force_lib "devel/sage/sage/structure/sage_object.pyx"

sage -t -force_lib "devel/sage/sage/misc/functional.py"

sage -t -force_lib "devel/sage/sage/modules/free_module_element.pyx"

sage -t -force_lib "devel/sage/sage/modular/modform/ambient_R.py"

sage -t -force_lib "devel/sage/sage/calculus/desolvers.py"

sage -t -force_lib "devel/sage/sage/calculus/calculus.py"

sage -t -force_lib "devel/sage/sage/calculus/tests.py"

sage -t -force_lib "devel/sage/sage/calculus/functional.py"

sage -t -force_lib "devel/sage/sage/calculus/interpolators.pyx"

sage -t -force_lib "devel/sage/sage/calculus/wester.py"

Some are probably due to ecl-11.1.1 so it should shrink, but it will take a bit of time since ecl, maxima and sage have to be rebuild.

kiwifb commented 13 years ago
comment:32

ecl-10.4.1, maxima-5.23.2:

sage -t --verbose -force_lib "devel/sage/sage/functions/special.py" times out because it stops:

Trying:
    elliptic_e(arccoth(Integer(1)), x**Integer(2)*e)###line 535:_sage_    >>> elliptic_e(arccoth(1), x^2*e)
Expecting:
    elliptic_e(arccoth(1), x^2*e)
The number 1 isn't in the domain of acoth
 -- an error. To debug this try: debugmode(true);

Error in format: Unknown format directive.
  The number ~:M isn't in the domain of ~A
               ^
while processing indirect format string:
  ~?
   ^
No restarts available.

Broken at LAMBDA.
MAXIMA>>
kiwifb commented 13 years ago
comment:33

Interesting one, notice that it still wants maxima-5.19.1

sage -t -force_lib "devel/sage/sage/interfaces/maxima_lib.py"
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima_lib.py", line 426:
    sage: t.limit(Ax=0,dir='above')
Exception raised:
    Traceback (most recent call last):
      File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/usr/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_0[93]>", line 1, in <module>
        t.limit(Ax=Integer(0),dir='above')###line 426:
    sage: t.limit(Ax=0,dir='above')
      File "expression.pyx", line 8202, in sage.symbolic.expression.Expression.limit (sage/symbolic/expression.cpp:31252)
      File "/usr/lib/python2.6/site-packages/sage/calculus/calculus.py", line 1122, in limit
        return l.sage()
      File "element.pyx", line 327, in sage.structure.element.Element.__getattr__ (sage/structure/element.c:2715)
      File "parent.pyx", line 277, in sage.structure.parent.getattr_from_other_class (sage/structure/parent.c:2841)
      File "parent.pyx", line 175, in sage.structure.parent.raise_attribute_error (sage/structure/parent.c:2636)
    AttributeError: 'sage.rings.integer.Integer' object has no attribute 'sage'
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima_lib.py", line 966:
    sage: maxima.version()
Expected:
    '5.19.1'
Got:
    '5.23.2'
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima_lib.py", line 1076:
    sage: maxima_version()
Expected:
    '5.19.1'
Got:
    '5.23.2'
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima_lib.py", line 639:
    sage: integrate(1/(x^3 *(a+b*x)^(1/3)),x)
Expected:
    Traceback (most recent call last):
    ...
    TypeError: Computation failed since Maxima requested additional constraints (try the command 'assume(a>0)' before integral or limit evaluation, for example):
    Is  a  positive or negative?
Got:
    Traceback (most recent call last):
      File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/usr/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_8[3]>", line 1, in <module>
        integrate(Integer(1)/(x**Integer(3) *(a+b*x)**(Integer(1)/Integer(3))),x)###line 639:
    sage: integrate(1/(x^3 *(a+b*x)^(1/3)),x)
      File "/usr/lib/python2.6/site-packages/sage/misc/functional.py", line 718, in integral
        return x.integral(*args, **kwds)
      File "expression.pyx", line 8153, in sage.symbolic.expression.Expression.integral (sage/symbolic/expression.cpp:30871)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 601, in integrate
        return indefinite_integral(expression, v)
      File "function.pyx", line 419, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:4486)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 85, in _eval_
        res = integrator(f, x)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/external.py", line 19, in maxima_integrator
        result = maxima.sr_integral(expression,v)
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 983, in sr_integral
        raise error
    RuntimeError: ECL says: Maxima asks:?mtext("Is  ",a,"  positive or negative?")
**********************************************************************
File "/usr/share/sage/devel/sage/sage/interfaces/maxima_lib.py", line 649:
    sage: integral(x^n,x)
Expected:
    Traceback (most recent call last):
    ...
    TypeError: Computation failed since Maxima requested additional constraints (try the command 'assume(n+1>0)' before integral or limit evaluation, for example):
    Is  n+1  zero or nonzero?
Got:
    Traceback (most recent call last):
      File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/usr/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_8[7]>", line 1, in <module>
        integral(x**n,x)###line 649:
    sage: integral(x^n,x)
      File "/usr/lib/python2.6/site-packages/sage/misc/functional.py", line 718, in integral
        return x.integral(*args, **kwds)
      File "expression.pyx", line 8153, in sage.symbolic.expression.Expression.integral (sage/symbolic/expression.cpp:30871)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 601, in integrate
        return indefinite_integral(expression, v)
      File "function.pyx", line 419, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:4486)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 85, in _eval_
        res = integrator(f, x)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/external.py", line 19, in maxima_integrator
        result = maxima.sr_integral(expression,v)
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 983, in sr_integral
        raise error
    RuntimeError: ECL says: Maxima asks:?mtext("Is  ",n+1,"  zero or nonzero?")
**********************************************************************
4 items had failures:
   1 of  95 in __main__.example_0
   1 of   3 in __main__.example_18
   1 of   4 in __main__.example_23
   2 of  11 in __main__.example_8
***Test Failed*** 5 failures.
For whitespace errors, see the file /home/francois/.sage/tmp/.doctest_maxima_lib.py
         [17.3 s]

----------------------------------------------------------------------
The following tests failed:

        sage -t -force_lib "devel/sage/sage/interfaces/maxima_lib.py"
Total time for all tests: 17.3 seconds

sage -t -force_lib "devel/sage/sage/symbolic/integration/integral.py times out because:

Trying:
    integrate(sin(x)*cos(Integer(10)*x)*log(x), x)###line 462:_sage_    >>> integrate(sin(x)*cos(10*x)*log(x), x)
Expecting:
    1/198*(11*cos(9*x) - 9*cos(11*x))*log(x) + 1/44*Ei(-11*I*x) - 1/36*Ei(-9*I*x) - 1/36*Ei(9*I*x) + 1/44*Ei(11*I*x)
Wrong number of arguments to gamma_incomplete
 -- an error. To debug this try: debugmode(true);

Error in format: Unknown format directive.
  Wrong number of arguments to ~:@M
                                  ^
while processing indirect format string:
  ~?
   ^
No restarts available.

Broken at LAMBDA.

sage -t -force_lib "devel/sage/sage/symbolic/relation.py" times out because of this:

Trying:
    solve([sqrt(x) + sqrt(y) == Integer(5), x + y == Integer(10)], x, y)###line 492:_sage_    >>> solve([sqrt(x) + sqrt(y) == 5, x + y == 10], x, y)
Expecting:
    [[x == -5/2*I*sqrt(5) + 5, y == 5/2*I*sqrt(5) + 5], [x == 5/2*I*sqrt(5) + 5, y == -5/2*I*sqrt(5) + 5]]
assoc: every list element must be an expression with two arguments; found: [sage147]
#0: simp_%solve(e=sage135,v=sage146,extraargs=[sage147])
 -- an error. To debug this try: debugmode(true);

Error in format: Unknown format directive.
  assoc: every list element must be an expression with two arguments; found: ~:M
                                                                               ^
while processing indirect format string:
  ~?
   ^
No restarts available.

sage -t -force_lib "devel/sage/sage/symbolic/assumptions.py":

Trying:
    decl2.assume()###line 89:_sage_    >>> decl2.assume()
Expecting:
    Traceback (most recent call last):
    ...
    ValueError: Assumption is inconsistent
declare: inconsistent declaration declare(x,odd)
 -- an error. To debug this try: debugmode(true);

Error in format: Unknown format directive.
  declare: inconsistent declaration ~:M
                                      ^
while processing indirect format string:
  ~?
   ^
No restarts available.

sage -t -force_lib "devel/sage/sage/symbolic/expression.pyx":

Trying:
    solve(acot(x),x)###line 7453:_sage_    >>> solve(acot(x),x)
Expecting:
    []
The number 0 isn't in the domain of cot
 -- an error. To debug this try: debugmode(true);

Error in format: Unknown format directive.
  The number ~:M isn't in the domain of ~A
               ^
while processing indirect format string:
  ~?
   ^
No restarts available.

sage -t -force_lib "devel/sage/sage/structure/sage_object.pyx" is interesting:

File "/usr/share/sage/devel/sage/sage/structure/sage_object.pyx", line 1053:
    sage: print "x"; sage.structure.sage_object.unpickle_all()
Expected:
    x...
    Successfully unpickled ... objects.
    Failed to unpickle 0 objects.
Got:
    x
    doctest:1: DeprecationWarning: Your word object is saved in an old file format since FiniteWord_over_OrderedAlphabet is deprecated and will be deleted in a future version of Sage (you can use FiniteWord_list instead). You can re-save your word by typing "word.save(filename)" to ensure that it will load in future versions of Sage.
    doctest:1: DeprecationWarning: Your word object is saved in an old file format since AbstractWord is deprecated and will be deleted in a future version of Sage (you can use FiniteWord_list instead). You can re-save your word by typing "word.save(filename)" to ensure that it will load in future versions of Sage.
    doctest:1: DeprecationWarning: Your word object is saved in an old file format since Word_over_Alphabet is deprecated and will be deleted in a future version of Sage (you can use FiniteWord_list instead). You can re-save your word by typing "word.save(filename)" to ensure that it will load in future versions of Sage.
    doctest:1: DeprecationWarning: Your word object is saved in an old file format since Word_over_OrderedAlphabet is deprecated and will be deleted in a future version of Sage (you can use FiniteWord_list instead). You can re-save your word by typing "word.save(filename)" to ensure that it will load in future versions of Sage.
    doctest:1: DeprecationWarning: ChristoffelWord_Lower is deprecated, use LowerChristoffelWord instead
     * unpickle failure: load('/home/francois/.sage/temp/vrooom/13483/dir_2/pickle_jar/_class__sage_interfaces_maxima_MaximaFunction__.sobj')
    Failed:
    _class__sage_interfaces_maxima_MaximaFunction__.sobj
    Successfully unpickled 585 objects.
    Failed to unpickle 1 objects.

sage -t -force_lib "devel/sage/sage/modules/free_module_element.pyx" now pass, so probably ecl-11.1.1 related.

sage -t -force_lib "devel/sage/sage/calculus/tests.py" (extracts):

File "/usr/share/sage/devel/sage/sage/calculus/tests.py", line 107:
    sage: integrate(log(1-x^2)/x, x)
Exception raised:
    Traceback (most recent call last):
      File "/usr/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/usr/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/usr/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_0[41]>", line 1, in <module>
        integrate(log(Integer(1)-x**Integer(2))/x, x)###line 107:
    sage: integrate(log(1-x^2)/x, x)
      File "/usr/lib/python2.6/site-packages/sage/misc/functional.py", line 718, in integral
        return x.integral(*args, **kwds)
      File "expression.pyx", line 8153, in sage.symbolic.expression.Expression.integral (sage/symbolic/expression.cpp:30871)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 601, in integrate
        return indefinite_integral(expression, v)
      File "function.pyx", line 419, in sage.symbolic.function.Function.__call__ (sage/symbolic/function.cpp:4486)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/integral.py", line 85, in _eval_
        res = integrator(f, x)
      File "/usr/lib/python2.6/site-packages/sage/symbolic/integration/external.py", line 19, in maxima_integrator
        result = maxima.sr_integral(expression,v)
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 977, in sr_integral
        return max_to_sr(maxima_eval(([max_integrate],[sr_to_max(SR(a)) for a in args])))
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 1228, in max_to_sr
        args.append(max_to_sr(car(max_args)))
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 1228, in max_to_sr
        args.append(max_to_sr(car(max_args)))
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 1228, in max_to_sr
        args.append(max_to_sr(car(max_args)))
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_lib.py", line 1221, in max_to_sr
        sage_expr=SR(maxima(expr))
      File "parent.pyx", line 915, in sage.structure.parent.Parent.__call__ (sage/structure/parent.c:6668)
      File "coerce_maps.pyx", line 156, in sage.structure.coerce_maps.NamedConvertMap._call_ (sage/structure/coerce_maps.c:4045)
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_abstract.py", line 1373, in _symbolic_
        return R(self._sage_())
      File "/usr/lib/python2.6/site-packages/sage/interfaces/maxima_abstract.py", line 1354, in _sage_
        maxima=self.parent())
      File "/usr/lib/python2.6/site-packages/sage/calculus/calculus.py", line 1646, in symbolic_expression_from_maxima_string
        raise TypeError, "unable to make sense of Maxima expression '%s' in Sage"%s
    TypeError: unable to make sense of Maxima expression 'li[2]' in Sage
File "/usr/share/sage/devel/sage/sage/calculus/tests.py", line 151:
    sage: integrate(ceil(x^2 + floor(x)), x, 0, 5)    # todo: Mathematica can do this
Expected:
    integrate(ceil(x^2) + floor(x), x, 0, 5)
Got:
    155/3
**********************************************************************
File "/usr/share/sage/devel/sage/sage/calculus/tests.py", line 189:
    sage: integrate(sin(x), x, a, b)
Expected:
    cos(a) - cos(b)
Got:
    ceil(cos(a))

sage -t -force_lib "devel/sage/sage/calculus/interpolators.pyx" is actually unrelated to maxima I think.

kcrisman commented 13 years ago
comment:34

Replying to @kiwifb:

ecl-10.4.1, maxima-5.23.2:

sage -t --verbose -force_lib "devel/sage/sage/functions/special.py" times out because it stops:

Trying:
    elliptic_e(arccoth(Integer(1)), x**Integer(2)*e)###line 535:_sage_    >>> elliptic_e(arccoth(1), x^2*e)
Expecting:
    elliptic_e(arccoth(1), x^2*e)
The number 1 isn't in the domain of acoth

Hmm, this must have to do with 5.23.2, because "Here arccoth doesn't have 1 in its domain, so we just hold the expression:" in the past. So Maxima must have improved something here.

    sage: integrate(ceil(x^2 + floor(x)), x, 0, 5)    # todo: Mathematica can do this
Expected:
    integrate(ceil(x^2) + floor(x), x, 0, 5)
Got:
    155/3

If this is right, that's good. No idea what's going on with the next one.

In general, I'm seeing a lot of stuff that's related to having the newer Maxima. I think that we should either upgrade Maxima first, and take care of things related to better error handling/functionality in Maxima, and then address this ticket separately, OR do this first with our current Maxima and then upgrade Maxima. My preference would be to upgrade Maxima first, assuming one can do that relatively easily (would ECL also have to be upgraded? It's annoying that one has to do them simultaneously at times, at least with some of my slower test machines.).

kiwifb commented 13 years ago
comment:35

I will do another run of tests with maxima-5.22.1 later. It is not too hard to go from one version of maxima to another. At least you don't have to rebuild anything else.

But it definitely looks like upgrading maxima would be a good thing, we'll have to make a ticket about it.

nbruin commented 13 years ago
comment:36

Something blows up rather badly here. The MAXIMA>> prompt indicates that you're dropped into to ECL debugger [the default behaviour of LISPs is to drop you into a debugger when an uncaught condition arises]. Ecllib tries to avoid this by executing any LISP code inside the LISP equivalent of try/except (being HANDLER-CASE). You can get tracebacks and code to see what is happening from the debugger (try ":h" to get help about the ECL debugger)

----------------------------------------------------------------------
| Sage Version 4.6.1, Release Date: 2011-01-11                       |
| Type notebook() for the GUI, and license() for information.        |
----------------------------------------------------------------------
Loading Sage library. Current Mercurial branch is: dev
sage: elliptic_e(arccoth(1), x^2*e)
The number 1 isn't in the domain of acoth
 -- an error. To debug this try: debugmode(true);

This is Maxima printing something on the STDOUT. We should be catching this before it gets printed! In other words, find which routine in maxima prints this and monkey-patch it to raise an error before printing this. Apparently, maxima does raise an error after printing this, which normally gets caught on maxima's top level. But since we decapitated maxima, we're getting that now, and that is where the next thing goes wrong:

Error in format: Unknown format directive.
  The number ~:M isn't in the domain of ~A
               ^
while processing indirect format string:
  ~?
   ^
No restarts available.

Broken at LAMBDA.
MAXIMA>> :b

Backtrace:
  > LAMBDA
  > print-object
  > common-lisp-user::sage-safe-apply

MAXIMA>> :le

(LAMBDA (CONDITION STREAM)
  (FORMAT STREAM
          "~?"
          (SIMPLE-CONDITION-FORMAT-CONTROL CONDITION)
          (SIMPLE-CONDITION-FORMAT-ARGUMENTS CONDITION)))

so a FORMAT statement is failing here. Apparently "Condition" objects (roughly "exception" objects in python) can have a format string and items hanging off them. FORMAT is LISPs "printf", but with a much more baroque set of options. FORMAT format specifiers have been rumoured to be Turing complete. Let's look at the parameters:

MAXIMA>> (SIMPLE-CONDITION-FORMAT-CONTROL CONDITION)

"The number ~:M isn't in the domain of ~A"
MAXIMA>> (SIMPLE-CONDITION-FORMAT-ARGUMENTS CONDITION)

NIL

So the "~:M" is probably a Maxima extension to format which is not accessible at the point where we are trying it. The fact that there are no arguments also indicates that Maxima might be forming a condition object that is not fit for general consumption.

Let's see where this error really arose by stepping further up in the backtrace:

MAXIMA>> :p

Broken at PRINT-OBJECT.
MAXIMA>> :le

(SI:LAMBDA-BLOCK PRINT-OBJECT
                 (SI::X STREAM)
                 (DECLARE (TYPE SIMPLE-CONDITION SI::X)
                  (SI::NO-CHECK-TYPE SI::X))
                 (IF *PRINT-ESCAPE*
                     (CALL-NEXT-METHOD)
                     ((LAMBDA (CONDITION STREAM)
                        (FORMAT STREAM
                                "~?"
                                (SIMPLE-CONDITION-FORMAT-CONTROL CONDITION)
                                (SIMPLE-CONDITION-FORMAT-ARGUMENTS CONDITION)))
                      SI::X STREAM)))
{{{
So this is LISP trying to print a CONDITION object. Apparently, Maxima has created a CONDITION object that doesn't allow this method to work. This could be a bug in ECL or it could be Maxima being non-compliant.
}}}
MAXIMA>> :p

Broken at COMMON-LISP-USER::SAGE-SAFE-APPLY.
MAXIMA>> :le

(SI:LAMBDA-BLOCK COMMON-LISP-USER::SAGE-SAFE-APPLY
                 (COMMON-LISP-USER::FUNC COMMON-LISP-USER::ARGS)
                 (DECLARE (SI::C-GLOBAL))
                 (HANDLER-CASE
                  (VALUES
                   (APPLY COMMON-LISP-USER::FUNC COMMON-LISP-USER::ARGS))
                  (SERIOUS-CONDITION (COMMON-LISP-USER::CND)
                   (VALUES NIL (PRINC-TO-STRING COMMON-LISP-USER::CND)))))
MAXIMA>> COMMON-LISP-USER::FUNC

MAXIMA-EVAL
MAXIMA>> COMMON-LISP-USER::ARGS

((MEVAL* '((MSETQ) $SAGE1 (($ELLIPTIC_E) ((%ACOTH) 1) ((MTIMES) ((MEXPT) $X 2) ((%EXP) 1))))))

Finally we're at the ecllib level. SAGE-SAFE-APPLY is the way ecllib tries to execute LISP code As you can see, the arguments here indicate that this is the original call we started with, translated to maxima (well, the underlying LISP representation of maxima). We know this raised an error, which gets processed by the HANDLER-CASE/SERIOUS-CONDITION. The PRINC-TO-STRING tries to make a string rendition of the CONDITION object, which fails for the reasons we have seen above.

So, the most direct way to solve the problem is to monkey-patch Maxima to raise an acceptable error instead of printing a message and sending a CONDITION object that crashes PRINC (which should be a near-universal printing routine). This should be relatively straightforward, because that Maxima routine already produces an acceptable string. Simply produce that string but instead of printing it, do an (ERROR string) or whatever is acceptable (the way "retrieve" gets monkey-patched in maximalib is already an example of that).

It does make me a bit pessimistic, though: Maxima improves some of its error reporting and in the process immediately blows up our error catching! This looks like something that is going to happen every time Maxima changes something. It's bad enough when dealing with the pexpect interface, but in this case we're basically using an unsupported and undocumented API! I know Robert Dodier is definitely sympathetic to making Maxima usable as a library, so perhaps if we can get reasonable functionality, he might be willing to take into account the required hooks when changing Maxima.

nbruin commented 13 years ago

Attachment: errorcatching.patch.gz

improve maxima-eval error reporting

nbruin commented 13 years ago

I have looked a little into the error reporting and it looks like maxima was not creating the error that crashed ecllib's error string creation routine -- instead it was maxima-eval, the maxima "top level" replacement we run to catch and interpret maxima's error reporting. Maxima's error reporting hasn't changed since 1985. It just happened that this was the first time we stumbled into an error message that was not properly handled. The new patch "errorcatching.patch" (to be applied after all the "rebased" patches) fixes this.

We now get:

sage: arccoth(1).simplify()
 -- an error. To debug this try: debugmode(true);
(result= MAXIMA-ERROR)
($error= ((MLIST) The number ~:M isn't in the domain of ~A 1 ACOTH))
TypeError: ECL says: The number 1 isn't in the domain of acoth
sage: elliptic_e(arccoth(1), x^2*e)
(result=
 (MAXIMA_EVAL
  . /usr/local/sage/4.6/local/share/maxima/5.22.1/share/orthopoly/orthopoly.lisp))
($error= ((MLIST) The number ~:M isn't in the domain of ~A 1 ACOTH))
(result= (MAXIMA_EVAL))
($error= ((MLIST) The number ~:M isn't in the domain of ~A 1 ACOTH))
 -- an error. To debug this try: debugmode(true);
(result= MAXIMA-ERROR)
($error= ((MLIST) The number ~:M isn't in the domain of ~A 1 ACOTH))
elliptic_e(arccoth(1), x^2*e)

So we see a bunch of things:

This should be closer to how sage handled this first.

nbruin commented 13 years ago

The following doctest from calculus.py now goes wrong:

limit(-e^(x^2)*erf(x) + e^(x^2), x=oo)

Which stumbles at calculus.calculus.limit at these lines:

   1122     return l.sage()
   1123     return ex.parent()(l)

Two return statements??? A little hg -blame points to #7661. Indeed, commenting out line 1122 fixes a lot of the limit problems. However, Burcin put in that line for a reason, I assume, so I'm hesitant to just commit a reversal.

Note that the issue of #7661 is exactly due to a namespace collapse from SR -> Maxima; something this interface could eventually solve much more elegantly, by just mapping (at no cost) SR.var('x') to the maxima identifier "sage_SR_var_x".

kiwifb commented 13 years ago
comment:39

two return statements is indeed strange. Is the second return ever reached? Could it be a left over forgotten there? Looking at the code I am not sure what ex.parent()(l) is supposed to be. I haven't had time to test the errorcatching patch but will try ASAP. One of my testing friend noted that maxima as a lib and ecl compiled with thread support are not compatible (ever seen a SIGPWR?). I am mentioning that, in case it becomes the default in ecl in the future.

nbruin commented 13 years ago
comment:40

two return statements is indeed strange. Is the second return ever reached?

No, it isn't. However, Burcin apparently thought that l.sage() would give better results. I'm actually thankful that he left in the original return statement because that served as a nice flag that something had changed there.

Could it be a left over forgotten there? Looking at the code I am not sure what ex.parent()(l) is supposed to be.

It is "Coerce l into the parent of ex", the original input parameter, i.e., coerce l into the Symbolic Ring. In the new code, "sr_limit" produces something that is hopefully coercible into the SymbolicRing -- it probably isn't a maxima object at all anymore. I think in all cases it's better to coerce into SR instead of calling l.sage(): On maxima objects it should be about the same and if limit produces something that cannot be represented in the SymbolicRing we should get an error. I'll ping him about it.

One of my testing friend noted that maxima as a lib and ecl compiled with thread support are not compatible (ever seen a SIGPWR?). I am mentioning that, in case it becomes the default in ecl in the future.

Good to know! A little googling found http://www.hpl.hp.com/personal/Hans_Boehm/gc/debugging.html It's the garbage collector that uses SIGXCPU and SIGPWR to synchronise cross-thread garbage collection. So if we want ecllib+threads we need to adjust the sage signal handler to properly handle those. BoehmGC is made for integration into C programs, so this may be doable.

kiwifb commented 13 years ago
comment:41

Re-running sage -testall with maxima-lib enabled, ecl-10.4.1 maxima-5.22.1, sage-4.6.2_alpha3. Most of the time out have disappeared expect for three which at first sight are unrelated, but don't fail without maxima-lib

sage -t  -force_lib devel/sage/sage/misc/randstate.pyx
sage -t  -force_lib devel/sage/sage/interfaces/psage.py
sage -t  -force_lib devel/sage/sage/interfaces/sage0.py

From the first one:

Trying:
    subsage = Sage()###line 134:_sage_    >>> subsage = Sage()
Expecting nothing
ok
Trying:
    s = ZZ(subsage('initial_seed()'))###line 135:_sage_    >>> s = ZZ(subsage('initial_seed()'))
Expecting nothing

And it hangs there.

I also have SIGFPE in the following

sage -t -force_lib "devel/sage/sage/functions/other.py"     *
sage -t  -force_lib devel/sage/sage/functions/log.py        *
sage -t  -force_lib devel/sage/sage/symbolic/constants.py
sage -t  -force_lib devel/sage/sage/symbolic/pynac.pyx 
sage -t  -force_lib devel/sage/sage/rings/complex_number.pyx 
sage -t  -force_lib devel/sage/sage/rings/real_double.pyx
sage -t  -force_lib devel/sage/sage/rings/real_mpfr.pyx 
sage -t  -force_lib devel/sage/sage/libs/mpmath/utils.pyx
sage -t  -force_lib devel/sage/sage/libs/mpmath/ext_libmp.pyx
sage -t  -force_lib devel/sage/sage/libs/mpmath/ext_impl.pyx
sage -t  -force_lib devel/sage/sage/libs/mpmath/ext_main.pyx
sage -t  -force_lib devel/sage/sage/calculus/calculus.py    *
sage -t  -force_lib devel/sage/sage/structure/element.pyx
sage -t  -force_lib devel/sage/sage/schemes/elliptic_curves/ell_point.py 
sage -t  -force_lib devel/sage/sage/schemes/elliptic_curves/ell_generic.py  *

The stars indicate where I also had some verbose out from maxima, usually ending with

($error= ((MLIST SIMP) No error.))

And there is quite few more. I am doing another run with the little correction in calculus.py to see if that solve some of them.

kiwifb commented 13 years ago
comment:42

scrap the three time out. They still happen without maxima-lib I was mistaken.